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PREFACE 


The  IDA  Paper  P-2300,  Joint  DoD/Industry  Study  on  Opportunities  in  Integrated 
Diagnostics,  is  intended  to  partially  fulfill  the  objective  of  Task  Order  T-B5-490, 
Integrated  Diagnostics  for  Weapons  and  Support  Systems.  It  documents  the  workshop 
case  studies  and  findings,  and  IDA’s  analyses  and  recommendations.  The  Opportunities 
in  Integrated  Diagnostic  workshops  were  hosted  by  IDA  for  OASD  (  P  &  L  )  on  21  &  22 
June  and  3  August,  1989  in  Alexandria,  Virginia. 

The  workshop  participants  were  asked  to  consider  some  examples  of  commercial 
applications  of  integrated  diagnostics,  and  then  to  (1)  identify  key  attributes  and  imple¬ 
mentation  approaches  used,  (2)  observe  which  attributes  and  implementation  approaches 
might  be  useful  to  address  DoD  weapons  system  maintenance  problems,  (3)  recommend 
how  DoD  might  validate  these  approaches,  and  (4)  consider  potential  obstacles  for  DoD 
application  of  these  diagnostic  improvement  opportunities. 

A  draft  of  this  paper  was  provided  to  all  of  the  workshop  participants  for  review 
and  comment.  We  would  like  to  thank  the  workshop  participants  for  contributing  their 
energy  and  talents  to  this  effort.  The  participants  are  listed  in  Appendix  A.  The  draft 
version  of  this  report  was  reviewed  within  the  Computer  and  Software  Engineering  Divi¬ 
sion  (CSED)  on  September  19,  1989,  by  R.  Wexelblat,  T.  Mayfield,  J.  Linn,  R.  Winner, 
and  J.  Pennell.  We  thank  Lew  Dimler  and  Betty  Pinna  for  the  excellent  organization  of 
the  workshops,  Betty  Pinna  for  typesetting  and  Katydean  Price  for  editing  the  report. 
Finally,  we  would  like  to  thank  Chris  Fisher  and  Marty  Meth  of  OASD  (P&L)  WSIG  for 
their  guidance  and  support. 


EXECUTIVE  SUMMARY 


Introduction- 

This  study  was  requested  by  the  Weapon  Support  Improvement  Group  (WSIG)  of 
the  Office  of  the  Assistant  Secretary  of  Defense  for  Production  and  Logistics 
(OASD(P&L)),  and  is  intended  to  identify  an  approach  to  take  advantage  of  opportuni¬ 
ties  for  improved  diagnostic  capability  already  demonstrated  in  selected  commercial 
industries.  This  report  documents  the  study  results  which  includes  the  opportunities 
identified  in  the  June  and  August,  1989  workshop  case  studies  and  findings,  and  IDA’s 
subsequent  analyses  and  recommendations. 

Workshop  General  Observation 

The  workshop  observed  that  companies  have  successfully  developed  and  imple¬ 
mented  diagnostic  elements  (e.g.,  built-in-test,  test  equipment,  technical  information  and 
training)  as  an  integrated  capability,  and  thereby  have  improved  diagnostic  accuracy, 
reduced  warranty  and  maintenance  costs,  and  increased  system  availability.  The  case 
studies  illustrated  dramatic  improvements  in  maintenance  productivity  and  operating 
efficiency:  (1)  The  General  Motors  (GM)  Computerized  Automotive  Maintenance  Sys¬ 
tem  (CAMS)  demonstrated  33%  improvement  in  maintenance  shop  productivity  within  3 
weeks  of  usage,  (2)  the  AT&T  5ESS  electronic  switching  system  automated  maintenance 
performance  exceeds  the  system  requirements  of  less  than  3  minutes  of  down  time  per 
year,  and  (3)  the  General  Electric  Ground-based  [aircraft]  Engine  Monitoring  (GEM) 
system  on-wing  (Lufthansa)  maintenance  performance  analyses  reduces  maintenance 
overhaul  budgets  by  5%  and  reduces  fuel  consumption  by  an  estimated  0.5%. 

DoD  Diagnostic  Needs 

Current  DoD  weapon  systems  field  maintenance  environments  require  large 
logistics  infrastructures  consisting  of  many  different  technical  specialists,  complex  test 
equipment,  and  voluminous  technical  information.  This  translates  to  vulnerable  and  bur¬ 
densome  forward  support.  Maintenance  depots  operate  with  imprecise  operational  per¬ 
formance  and  diagnostic  technical  information  from  other  maintenance  activities.  Field 
and  depot  maintenance  workloads  are  a  direct  function  of  diagnostic  accuracy.  A  number 
of  assessments  on  weapon  systems  support  have  documented  the  difficulty  in  performing 
accurate  diagnostics  on  DoD  systems  (the  lange  of  maintenance  actions  resulting  in  the 
removal  of  items  with  no  evidence  of  failure  frequently  varies  between  twenty  and  fifty 
percent). 


Future  DoD  requirements  call  for  increased  mobility,  small  deployments  with 
increased  sustainability  and  the  potential  for  increased  vulnerability.  These  operational 
needs  can  be  addressed  by  reducing  the  forward  burden  of  spares  and  diagnostic  equip¬ 
ment.  Improved  diagnostic  accuracy  will  reduce  the  burden  of  diagnostic  equipment  and 
spares  as  will  the  requirements  for  the  many  technical  specialists. 

Summary  of  Recommendations 

The  workshop  case  studies  highlighted  a  number  of  key  attributes  that  facilitated 
successful  application  of  integrated  diagnostics.  The  resulting  payoffs  demonstrated  in 
the  commercial  implementations  are  lower  costs  to  organizations  (warranty),  quicker 
repair  response  times  (customer  satisfaction),  improved  repair  accuracy  (more  up-time), 
and  enhanced  feedback  to  design  for  next  generation  (future  support  cost  avoidance  and 
more  rapid  diagnostic  maturity).  Interestingly,  the  key  attributes  that  facilitated  the 
application  in  the  commercial  cases  often  contrast  with  the  perception  of  current  DoD 
practices  and  guidelines.  The  following  recommendations  are  based  on  this  observation 
and  are  divided  into  three  areas:  near  term  actions  to  identify  new  opportunities  by 
focusing  on  contrasting  commercial  and  DoD  diagnostic  practices,  long  term  actions  to 
develop  a  planning  framework  that  will  assist  DoD  in  the  implementation  of  applicable 
integrated  diagnostics  concepts,  and  recommendations  resulting  from  the  general 
workshop  consensus. 

a.  Near  Term  Actions:  The  Services  collect  huge  amounts  of  information  relating 
to  systems  performance,  use,  and  maintenance  for  a  variety  of  reasons  that 
are  not  directly  tied  to  diagnostic  needs.  DoD  should  initiate  a  broad  based 
study  effort  to  investigate  DoD’s  practices  and  compare  them  with  what  was 
identified  in  the  commercial  cases: 

(1)  Identify  what  information  relating  to  maintenance  is  collected,  assess  how 
it  is  used,  identify  additional  data  opportunities,  and  assess  how  DoD 
should  capture  and  analyze  data. 

(2)  Assess  how  maintenance  diagnostic  responsibilities  are  assigned  and  per¬ 
formed  within  organizational  structures. 

(3)  Identify  the  types  and  capabilities  of  information  links  among  various  diag¬ 
nostics  elements. 
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(4)  Examine  approaches  to  data  collection  that  make  the  work  load  associ¬ 
ated  with  the  collection  process  transparent  to  maintenance  personnel  and 
identify  opportunities  to  further  the  application  of  integrated  diagnostics 
concepts. 

b.  Long  Term  Actions:  Based  on  the  results  of  the  near  term  actions,  investigate 
implementation  and  environmental  issues  that  relate  to  the  uniqueness  of 
DoD  type  missions: 

(1)  Assess  the  usefulness  of  applying  the  capability  of  using  data  that  is 
rapidly  fed-back  from  maintenance  to  influence  design  and  system  evolu¬ 
tion  and  assess  how  DoD  might  go  about  implementing  such  a  concept. 

(2)  Develop  a  planning  framework  that  considers  the  uniqueness  of  DoD’s 
needs  and  facilitates  the  implementation  of  integrated  diagnostics  con¬ 
cepts. 

c.  General  Workshop  Consensus:  The  workshop  participants  concluded  that  the 
integration  of  diagnostics  elements  provides  a  unique  opportunity  for  DoD  to 
significantly  enhance  the  maintenance  and  availability  of  today’s  systems  and 
unanimously  recommended  that  DoD  undertake  the  initiatives  to  demonstrate 
and  communicate  opportunities  in  integrated  diagnostics: 

(1)  Accomplish  demonstrations  on  fielded  systems  to  further  evaluate  as  well 
as  learn  of  application  opportunities  for  integrated  diagnostic  concept 
payoffs. 

(2)  Expand  efforts  to  increase  the  awareness  and  understanding  of  integrated 
diagnostics  opportunities  by  presenting  information  to  senior  level  using 
and  maintenance  communities. 
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1.  INTRODUCTION 

1.1  PURPOSE 


A  study  was  conducted  under  Task  Order  T-B5-490  for  the  Weapon  Support 
Improvement  Group  (WSIG),  Office  of  the  Assistant  Secretary  of  Defense  for  Production 
and  Logistics  (OASD(P&L))  to  identify  DoD  opportunities  in  integrated  diagnostics. 
The  objectives  of  the  study  were  to  (1)  provide  a  forum  to  present  and  discuss  advanced 
industrial  and  DoD  integrated  diagnostics  applications,  (2)  establish  a  consensus  on  the 
specific  integrated  diagnostics  applications  that  have  the  potential  of  addressing  DoD’s 
systems  maintenance  problems,  and  (3)  address  the  research,  feasibility,  and 
implementation  issues. 

1.2  WORKSHOP  ORGANIZATION 

This  study  included  two  workshop  meetings  hosted  by  IDA.  The  workshop 
participants  were  carefully  selected  for  their  in-depth  technical  background  and  personal 
experience.  These  participants  represented  (in  almost  equal  proportions)  the  following 
areas:  (1)  DoD  maintainers,  (2)  DoD  R&D/acquisition  managers,  (3)  Defense 
industry,  and  (4)  commercial  industry.  Appendix  A  provides  a  list  of  workshop 
participants. 

The  first  workshop  was  conducted  on  June  21-22,  1989.  Mr.  Martin  Meth, 
Director  WSIG,  led  off  the  discussions  with  an  introductory  presentation  (contained  in 
Appendix  B)  highlighting  the  concept  and  DoD  strategy  for  integrated  diagnostics. 
Industry  and  DoD  case  studies  followed  and  were  intended  to  focus  the  participants’ 
attention  on  potential  opportunities  to  apply  and  accelerate  the  use  of  integrated 
diagnostics  in  systems  applications. 

The  workshop  participants  were  asked  to  consider  some  examples  of  commercial 
applications  of  integrated  diagnostics,  and  then  to  (1)  identify  key  attributes  and 
implementation  approaches  used,  (2)  observe  which  attributes  and  implementation 
approaches  might  be  useful  to  address  DoD  weapons  system  maintenance  problems,  (3) 
recommend  how  DoD  might  validate  these  approaches,  and  (4)  consider  potential 
obstacles  for  DoD  application  of  these  diagnostic  improvement  opportunities.  The 
participants  also  were  divided  into  four  working  groups  and  were  asked  to  focus  on  these 
questions  from  the  perspectives  of  the  following  communities: 


a.  Users  and  supporters 

b.  Acquisition  and  system  development 

c.  Engineering  technical  data  and  standards 

d.  Research  and  development 

The  second  workshop  was  conducted  on  August  3,  1989.  This  workshop  was 
specifically  intended  to  review  a  preliminary  draft  of  this  workshop  paper  and  to  continue 
the  consensus  building  from  the  first  workshop. 
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2.  BACKGROUND 

2.1  INTRODUCTION 


Diagnostics  is  the  practice  of  investigating  the  cause  or  nature  of  specific 
problems  that  inhibit  normal  system  operation.  Today’s  system  diagnostic  capability  is 
developed  and  provided  through  engineering  design,  testing,  technical  information,  and 
trained  personnel.  The  diagnostic  elements  that  support  this  capability  include  built-in¬ 
test  (BIT),  automatic  and  manual  test  equipment,  integration  support  facilities  based  on 
elements  of  weapons  systems  and  external  support  equipment,  and  limited  condition 
monitoring.  The  diagnostic  technical  information  is  provided  to  the  maintainers  in  the 
form  of  technical  manuals  and  to  a  lesser  extent  through  limited  capability  information 
delivery  systems. 

2.2  ACQUISITION  AND  SUPPORT  ISSUES 

Maintenance  depots  operate  with  little  feedback  of  maintenance  data  to  allow 
system  wide  improvement  of  the  diagnostic  process.  Across  DoD,  maintainers 
compensate  for  diagnostic  accuracy  problems  with  trial-and-error  component 
replacement  techniques.  A  DoD  Task  Force  on  Productivity  in  Support  Operations 
(1986)  discovered  that  the  range  of  avionics  maintenance  actions  resulting  in  removal  of 
items  with  no  evidence  of  failure  varied  between  twenty  and  fifty  percent. 

DoD’s  current  approach  for  acquiring  and  using  diagnostic  capabilities  is  not 
focused  only  on  the  operational  objectives  of  the  weapon  system,  but  also  many  other 
objectives  which  create  the  following  conditions: 

a.  Diagnostic  specifications  are  not  clearly  derived  from  operational  objectives. 

b.  Separate  diagnostic  element  specifications  are  often  prepared  and  put  on 
contract  by  different  organizations. 

c.  Separate  maintenance  organizations  independently  control  the  use  of 
diagnostic  capabilities. 

2.3  INTEGRATED  DIAGNOSTIC  CONCEPT 

Integrated  diagnostics  concept  requires  the  design  and  development  of  these 
diagnostic  elements  as  a  “package”  for  system  maintenance.  This  differs  from  the  current 
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DoD  practice  of  specifying  and  buying  isolated  diagnostic  elements.  The  concept  of 
acquiring  diagnostics  capability  as  a  package  evolved  out  of  initiatives  from  the 
Automatic  Test  Committee  and  Logistics  Management  Committee  of  the  National 
Security  Industrial  Association  (NSIA).  The  primary  focus  of  this  effort  was  to  improve 
maintenance  quality  through  discipline  in  diagnostics  design  and  development. 

NSIA’s  involvement  spans  the  last  four  years  and  includes  three  NSIA-sponsored 
conferences  that  defined  the  problems  and  advanced  the  integrated  diagnostics  initiative. 
The  NSIA  approach  has  been  to  get  specialists  involved  by  forming  an  Integrated 
Diagnostics  Working  Group  with  representatives  from  200  companies  along  with  DoD 
counterparts  in  each  of  the  various  task  areas.  Out  of  these  efforts  evolved  an  NSIA 
Model  Statement  of  Work  (SOW)  for  integrated  diagnostics  which  places  the 
responsibility  for  diagnostic  capabilities,  performance,  and  integration  on  the  contractor. 
The  primary  goal  of  the  Model  SOW  is  to  deliver  a  system  where  the  individual  diagnostic 
elements  are  specifically  designed  to  complement  each  other. 

NSIA  also  sponsored  a  series  of  three  DoD  and  Industry  “Executive 
Roundtables”  in  1988.  In  attendance  were  40  executives  from  20  companies  (including  11 
vice  presidents,  10  directors  and  5  program  managers)  as  well  as  12  Flag  Officers  and 
Senior  Executive  Service  (SES)  members  from  DoD.  The  Roundtable  participants 
agreed  that  the  new  acquisition  approach  is  better  than  the  current  method  of  acquiring 
diagnostics,  that  the  NSIA  Model  SOW  is  a  good  starting  point,  and  that  DoD  and 
Industry  should  begin  to  apply  the  concept  now. 

2.4  LONG  TERM  OBJECTIVES  AND  NEAR  TERM  STRATEGY 

This  workshop,  as  well  as  the  joint  DoD  and  Industry  participation  in  the  NSIA 
meetings,  are  part  of  an  OASD(P&L)  WSIG  strategy  to  achieve  three  long-term 
objectives:  (1)  to  improve  the  accuracy  of  maintenance  actions,  (2)  to  make  diagnostic 
technology  improvement  transparent  to  the  weapon  system  maintainer,  and  (3)  to 
significantly  reduce  the  logistics  “deployment  tail”  (manuals,  test  equipment,  spare  parts, 
manpower  specialization,  etc.). 

In  addition  to  the  acquisition  aspect  of  the  WSIG  strategy,  a  second  part  of  this 
strategy  is  to  stimulate  the  fielding  of  mature  technology  solutions  in  weapon  systems  by 
demonstrating  the  feasibility  of  already  developed  laboratory  diagnostics  technology 
programs.  Towards  this  end,  a  new  FY-90  Research  and  Development  program  element 
(603708D),  Integrated  Diagnostics  Technology  Demonstration  Program,  was  established 
by  the  Defense  Resources  Board. 


The  objective  of  the  Technology  Demonstration  Program  is  to  prove  the  feasibility 
of  implementing  integrated  diagnostics  concepts  and  to  measure  potential  improvements 
by  conducting  large-scale  field  demonstrations  on  operational  weapon  systems.  Several 
laboratory  diagnostic  research  projects  were  selected  based  on  the  following  criteria: 

a.  Each  project  demonstrates  DoD  research  that  is  ready  for  application. 

b.  The  project  proposes  a  lead  weapon  system  to  demonstrate  the  generic 
diagnostic  technology  and  is  managed  by  the  program  manager. 

c.  The  demonstration  project  has  the  feasibility  of  fostering  cross-Service 
applications  of  the  demonstrated  diagnostic  technology. 

d.  The  proposed  diagnostic  technology  demonstrations  must  have  user 
commitment  to  implement  weapon  system  diagnostic  improvements. 
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3.  CASE  STUDIES 

A  review  of  commercial  product  maintenance  practices  in  preparation  for  this 
study  revealed  that  faced  with  increasing  product  complexity,  companies  began  to 
identify  and  address  their  diagnostic  needs  as  an  intentionally  integrated  “package”  of 
capability.  For  this  workshop,  case  histories  and  studies  (predominantly  from  the 
commercial  sector)  were  selected  to  illustrate  (1)  why  and  how  industry  was  doing  this, 
(2)  what  the  implications  might  be  in  terms  of  DoD’s  maintenance  environment,  and  (3) 
how  they  might  relate  to  DoD  integrated  diagnostics  initiatives  already  underway. 

A  broad  spectrum  of  case  studies  was  selected  and  included  examples  from  the 
automotive,  aviation,  communications,  computer,  and  power  industries.  The 
presentations  were  selected  as  examples  of  one  or  more  of  the  following  integrated 
diagnostics  approaches: 

•  Diagnostic  elements  developed  as  a  package  and  designed  to  work  together. 

•  Effective  use  of  engineering  diagnostic  information  through  the  application  of 
transparent  to  maintainer  operational  and  support  information  capture,  analysis, 
update,  and  feedback  for  improved  diagnostic  system  performance. 

•  New  computer-aided  engineering  (CAE)  design  and  analysis  tools  that  support 
diagnostic  system  effectiveness  concurrent  with  product  design.. 

•  New  on-board  condition  monitoring  sensors  to  improve  accuracy  and  timeliness 
of  maintenance  actions. 

The  objective  of  the  presentations  was  to  provide  examples  of  diagnostic 
capabilities  that  have  been  developed  and  applied  in  an  integrated  diagnostic  approach. 
The  presentations  were  intended  to  show  how  these  approaches  were  exploited,  what 
technologies  were  applied,  and  provide  some  insight  as  to  the  potential  benefits  for  the 
users  and  maintainers.  These  perspectives  set  the  stage  for  addressing  the  following 
areas  during  the  working  group  sessions:  (1)  identify  key  attributes  and  implementation 
approaches  used,  (2)  observe  which  attributes  and  implementation  approaches  might  be 
useful  to  address  DoD  weapons  system  maintenance  problems,  (3)  recommend  how  DoD 
might  validate  these  approaches,  and  (4)  consider  potential  obstacles  for  DoD 
application  of  these  diagnostic  improvement  opportunities. 
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3.1  CASE  STUDY  ATTRIBUTES 


The  following  summarizes  the  typical  attributes  of  the  eight  case  studies.  These 
attributes  have  been  grouped  under  the  general  categories  of  Diagnostics  Requirements, 
Organizational  Accountability,  Implementation  Characteristics,  and  Operational 
Characteristics.  Summaries  of  the  individual  case  studies  as  well  as  a  listing  of  the  system 
attributes  and  general  factors  that  influenced  the  diagnostic  successes  in  the  respective 
cast  studies  are  located  in  Appendix  C. 


3.1.1  Diagnostic  Requirements 

Each  of  the  case  studies  presented  was  initiated  and  justified  based  on  high-level, 
diagnostic-related  support  and  system  performance  requirements.  These  requirements 
were  traceable  to  areas  in  which  diagnostics  enhancements  could  lead  to  a  positive 
improvement  (i.e.,  cost  of  repair,  system  availability,  warranty  impacts,  spares  burden, 
technical  manuals  and  training,  etc.).  Common  diagnostic  requirement  characteristics 
include  the  following: 


•  Diagnostic  performance  requirements  were  defined  in  operational  terms  (as 
opposed  to  specification  requirements  like  false  alarm  rates)  and  the 
implementation  of  these  requirements  was  left  to  the  equipment  design 
organization. 


•  Requirements,  which  were  established  by  one  corporation  and  then  passed  on  to 
another  to  develop  the  diagnostics  capability,  took  the  form  of  broadly  stated 
measurable  maintenance  and  support  metrics  or  functional  capabilities,  (i.e., 
reduced  volume  for  technical  manuals,  higher  levels  of  availability,  level  of 
repair). 


•  Requirements  that  were  established  internally  by  the  corporation  (or  from  within 
the  corporation’s  own  operations  organization)  generally  were  focused  on  the  cost 
and/or  burden  of  providing  committed  services  or  product  performance  needs 
over  some  sustained  period  of  system  life.  These  needs  were  determined  from 
market  analyses,  warranties,  availability,  overhead,  or  cost  containment 
requirements. 

3.1.2  Organizational  Accountability 


In  the  majority  of  the  case  studies,  a  single  organization  was  responsible  for  the 
diagnostic  system  throughout  its  life  cycle  from  concept  definition  through  the  design 
evolution,  installation,  and  support  of  field  applications.  Typically  the  organizations 
exhibited  the  following  characteristics: 
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•  A  single  organization  was  tied  to,  and  held  authority  and  responsibility  for,  the 
delivered  and  evolving  maintenance  capabilities. 

•  A  single  organization  had  dedicated  resources  (dollars,  people,  and  facilities)  to 
improve  diagnostic  capability  across  organizational  and  functional  domains. 

•  A  single  organization  was  committed  to  improve  and  evolve  the  integrated 
diagnostic  capabilities  beyond  the  initial  installation  and  fielding  of  the  host 
system. 

3.1.3  Implementation  Characteristics 

The  case  studies  highlighted  a  number  of  common  key  integrated  diagnostic 
implementation  characteristics.  These  implementation  characteristics  including 
methodologies  and  approaches  very  often  crossed  over  diagnostic  element  boundaries 
and  depended  on  analyses  of  maintenance  information  for  the  integrated  diagnostics 
system  maturation  and  evolution.  Typically  the  key  implementation  characteristics 
include  the  following: 

•  The  diagnostic  system  designers  were  not  restricted  by  the  classical  maintenance 
and  support  boundaries  and  developed  diagnostic  capabilities  that  were 
integrated  across  functional  lines  (i.e.,  BIT,  fault  logging,  data  capture,  data  and 
trend  analysis,  prognostics,  presentation  of  maintenance  instructions,  training, 
etc.). 

•  The  implementation  approaches  mitigated  the  potential  of  human  error  by 
making  transparent  to  the  maintainer  the  maintenance  information  collection, 
and  providing  improved  tools  for  automating  analysis  and  presenting  information 
to  the  maintainer. 

•  The  diagnostic  system  decision  used  BIT  and/or  functional  (on-board  equipment) 
sensors  to  record  system  performance  and  operational  status. 

•  The  diagnostic  system  design  established  an  implementation  environment  that 
called  for  conducting  maintenance  information  analyses  off-equipment  at  a 
centralized  location. 

•  The  diagnostic  approach  used  flexible,  intelligent  electronic  presentation 
technologies  to  select  and  display  maintenance  and  support  information. 

•  The  system  designers  implemented  diagnostic  elements  with  existing  enabling 
technologies  as  opposed  to  pushing  state-of-the-art  technologies. 
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•  Implementations  focused  on  a  closed  loop  maintenance  information  feedback 
capability,  generally  through  a  centralized  organization,  to  evolve  and  rapidly 
mature  diagnostics. 

•  The  information  feedback  approach  stimulated  the  user  to  input  data  on  system 
performance. 

•  System  designers  used  methods  and  tools  which  allowed  concurrent  assessment  of 
diagnostic  characteristics  during  design  cycle. 

3.1.4  Operational  Characteristics 

The  case  studies  summarized  a  common  set  of  key  operating  characteristics  that 
were  essential  to  maintenance  acceptance,  diagnostic  accuracy,  and  system 
jupportability. 

•  The  integrated  diagnostics  systems  made  extensive  use  of  centralized  analyses 
facilities  that  permitted  the  rapid  analysis  and  update  of  critical  maintenance 
information  and  ensured  early  and  continued  design  feedback  data  essential  for 
maintenance  maturation. 

•  Acceptance  by  the  maintenance  community  was  enhanced  by  simplified 
maintenance  information  input  techniques  (e.g.,  no  numerical  codes,  transparent 
data  collection,  and  interactive  electronic  maintenance  and  support  instructions). 

•  The  integrated  diagnostic  systems  typically  provided  automated  configuration 
management  of  product  and  support  tools. 

•  The  maintainer  was  provided  a  flexible  interface  and  set  of  functions,  which 
allowed  the  maintainer  to  improvise  effective  maintenance  solutions  within 
bounded  limits. 
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4.  COMPARISONS  OF  TYPICAL  CASE  STUDY  ATTRIBUTES  AND 
DOD  PRACTICES 

The  following  tables  compared  the  typical  diagnostic  related  attributes  found  in 
the  case  studies  with  typical  diagnostics  related  DoD  practices.  Tables  1  through  4 
highlight  these  observations.  Table  1  compares  how  the  diagnostic  requirements  are 
typically  defined  and  translated  into  system  applications.  Table  2  addresses  the  typical 
differences  in  the  organizational  accountability  for  developing  and  evolving  the  system 
diagnostic  elements  over  the  product  life  cycle.  Table  3  focuses  on  the  typical  diagnostics 
related  implementation  characteristics  found  in  the  case  studies  and  observed  in  DoD 
practices.  Table  4  summarizes  the  key  operating  characteristics  found  in  the  case  studies 
and  compares  them  with  the  observed  typical  DoD  operationally  related  maintenance 
and  support  practices. 


Table  1.  Definition  and  Translation  of  System  Diagnostics  Requirements 


Typical 

Case  Study  Attributes 


Started  with  High-Level  Diagnostic  Related 
System  Performance  Requirements  (e.g., 
availability,  cost  of  warranty  spares) 

•  Requirements  are  stated  in 
operational  terms  and 
implementation  is  left  to  designer. 

•  Requirements  are  determined  by 
customer  needs  or  by  the  system 
developer  needs  to  provide  a 
competitive  service. 


Observations  of 
Typical  DoD  Practices 


DoD  Typically  States  Very  General 
Diagnostics  Goals  in  Strategic  Plans  (e.g., 
R&M  2000  to  reduce  burden  by  50%)  and 
Over  Specifies  Solutions  in  Contracts 

•  DoD  often  specifies  requirements, 
by  reference  to  standards  that 
over  constrain  designer’s  solution 
while  striving  for  related  system 
compatibility. 

•  DoD  contractors  are  often  not 
contractually  bound  to  deliver 
maintenance  and  supportability 
performance. 

•  Acquisition  of  diagnostic 
requirements  are  often  delayed 
until  after  production  delivery. 

•  Operational  diagnostic  needs  are 
seldom  translated  into  contractual 
terms. 
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Table  2.  Organizational  Accountability  for  Developing  and  Evolving  Diagnostics  Over 
Product  Life  Cycle 


Typical 

Case  Study  Attributes 

Observations  of 

Typical  DoD  Practices 

Single  Organization  Accountable  for 
Diagnostic  Capability  Throughout 

Product  Life  Cycle 

•  Held  authority  and  responsibility 
for  diagnostic  performance  in  the 
field. 

•  Implemented  an  improved 
diagnostic  system  across 
organizational  boundaries. 

•  Evolved  diagnostic  system  beyond 
initial  installation. 

Accountability  for  Diagnostics 

Distributed  Across  Functional  Elements 

•  DoD  maintenance  responsibilities 
fragmented  across  multiple 
functional  organizations. 

•  Diagnostic  elements  usually 
acquired  and  managed  separately. 

•  Developer  involvement  in 
diagnostics  maturation  often  end 
with  delivery. 

•  Multiple  contractors  usually 
developed  discrete  diagnostic 
elements  and  DoD  then  took  over 
management. 
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Table  3.  Typical  Diagnostics-Related  Implementation  Characteristics 


Typical 

Case  Study  Attributes 

Observations  of 

Typical  DoD  Practices 

Key  Implementation  Characteristics  of 
Integrated  Diagnostic  Systems 

•  System  designers  not  restricted  by 
classical  maintenance  boundaries. 

•  System  designers  used  advanced 
CAE  tools  to  support  concurrent 
analysis  of  expected  diagnostic 
performance. 

•  Mitigated  human  error  by 
automating  maintenance 
collection. 

•  Set  out  to  build  a  closed  loop 
diagnostic  system. 

•  Used  existing  technology. 

•  Capitalized  on  existing  functional 
sensors  and  built-in-test 
hardware. 

•  Recorded  malfunctions  on- 
equipment. 

•  Analyzed  malfunctions  off- 
equipment. 

•  Used  intelligent  electronic  display 
of  maintenance  information. 

•  Transparently  captured 
maintenance  history  information. 

•  Minimized  on-equipment  software 
changes. 

DoD  Implementation  Characteristics 

•  DoD  has  antiquated,  error  prone, 
maintenance  collection  and 
feedback  capabilities  that  are  not 
well  integrated  across  functional 
elements. 

•  DoD  and  it’s  contractors  lack  the 
integration  of  CAE  tools  across 
weapon  system  design  to  support 
concurrent  analysis  of  diagnostic 
performance  during  design. 

•  DoD  lacks  centralized  data 
analyses  centers  focusing  on 
diagnostic  accuracy. 

•  Routine  update  of  maintenance 
and  diagnostic  data  is  slow. 

•  Design  feedback  is  limited  and 
slow. 

•  DoD  maintenance  and  support 
functions  are  paper  intensive. 

•  DoD  focuses  on  discrete 
diagnostic  elements. 

Table  4.  Summary  of  Key  Diagnostics-Related  Operating  Characteristics 


Typical 

Case  Study  Attributes 

Observations  of 

Typical  DoD  Practices 

Key  Operating  Characteristics  of 

Integrated  Diagnostic  System 

•  Central  analysis  facility  for  rapid 
analysis  and  update  of  diagnostic 
information. 

•  Simplified  maintainer  input. 

•  Automated  configuration 
management  of  product  and 
support  tools. 

•  Flexible  systems  allow 
maintenance  personnel  to 
improvise  solutions  within 
bounded  structure. 

•  Integrated  supporting 
communications  systems. 

DoD  Generally  Delegates  the  Operation 
and  Up-Date  of  Discrete  Diagnostic 
Elements  to  Multiple  Organizations 
(technical  manuals,  test  equipment, 
on-board  sensors  &  BIT,  training) 

•  DoD  maintenance  data  collection 
techniques  are  paper  intensive 
and  often  originate  from  verbal 
inputs. 

•  DoD  configuration  management 
and  maintenance  data  traceability 
to  configuration  items  are 
significantly  less  capable  than  the 
case  study  examples. 

•  DoD  captures  much  logistics  data 
but  it  is  often  analyzed  in  multiple 
organizations  and  is  not  well 
integrated  across  functions. 

•  DoD  diagnostic  capabilities  must 
be  tolerant  of  potential 
communications  shutdowns  during 
wartime  conditions. 

5.  RECOMMENDATIONS 


The  workshop  discussions  clearly  illustrated  that  today’s  systems  (both 
commercial  and  military)  are  faced  with  increasing  maintenance  and  support  burdens. 
These  increased  burdens  carry  with  them  an  increased  need  for  effective  diagnostics. 
This  need  is  tied  directly  to  customer  requirements  for  sustained  product  quality  and  cost 
effective  maintenance.  Payoffs  demonstrated  in  the  commercial  implementations  of 
integrated  diagnostics  are  lower  costs  to  organizations  (warranty),  quicker  repair 
response  times  (customer  satisfaction),  repair  accuracy  (more  up-time),  and  enhanced 
feedback  to  design  for  next  generation  (future  support  cost  avoidance  and  more  rapid 
diagnostic  maturity) . 

The  workshop  cases  studies  highlighted  a  number  of  key  attributes  (See  Section  4)  that 
facilitated  successful  application  of  integrated  diagnostics  concepts  in  commercial  cases. 
Interestingly,  these  key  attributes  often  contrast  with  the  perception  of  current  DoD 
practices  and  guidelines.  The  following  recommendations  are  based  on  these 
observations  and  are  divided  into  three  areas:  near  term  actions  to  identify  new 
opportunities  by  focusing  on  contrasting  commercial  and  DoD  diagnostic  practices,  long 
term  actions  to  develop  a  planning  framework  that  will  assist  DoD  in  the  implementation 
of  applicable  integrated  diagnostics  concepts,  and  recommendations  resulting  from  the 
general  workshop  consensus. 

5.1  NEAR  TERM  ACTIONS 

At  present  the  Services  collect  huge  amounts  of  information  relating  to  systems 
performance,  use,  and  maintenance  for  a  variety  of  reasons  that  are  not  directly  tied  to 
system  diagnostic  needs.  Specific  near  term  actions  are  needed  to  identify  what  data  is 
collected  and  how  this  information  is  used,  assess  if  this  information  may  be  used  to 
enhance  diagnostics  performance,  identify  if  other  information  is  needed,  and  assess  how 
the  DoD  maintenance  data  collection  practices  compare  with  the  commercial  cases 
studied. 


a.  DoD  should  initiate  a  broad  based  study  effort  to  identify  what  information 
relating  to  maintenance  actions  is  collected  within  DoD  operational 
organizations.  The  study  should  assess  how  this  information  is  used  to 
address  diagnostic  needs  as  well  as  identify  opportunities  where  DoD  could 
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better  apply  available  information.  The  study  also  should  focus  on  what 
additional  data  would  be  useful  and  how  DoD  should  capture,  analyze  and 
apply  this- information.  Finally,  the  study  should  compare  these  current  DoD 
maintenance  data  collection  practices  with  what  was  seen  in  the  commercial 
cases. 

b.  DoD  should  identify  how  maintenance  responsibilities  (across  the  spectrum 
from  development  of  a  system  capability,  to  performing  maintenance,  to 
modifying  and  managing  a  fielded  capability)  are  assigned  and  carried  out  in 
today’s  DoD  organizational  structures.  The  study  should  assess  what  are  the 
limitations  of  these  organizational  structures,  and  compare  these  structures  to 
what  was  seen  in  the  commercial  cases. 

c.  DoD  should  assess  the  types  and  capabilities  of  information  links  among 
various  DoD  diagnostic  elements  (at  all  maintenance  levels)  and  compare 
these  information  links  with  what  was  seen  in  the  commercial  cases.  The 
study  also  should  identify  specific  diagnostic  payoff  and  improvement 
opportunities  that  would  result  from  enhanced  information  links. 

d.  DoD  should  examine  the  various  approaches  to  maintenance  data  collection 
that  make  the  work  load  and  burden  associated  with  data  collection  virtually 
transparent  to  the  maintenance  personnel.  The  near  term  study  should  assess 
this  transparency  issue  as  applied  in  DoD  and  as  compared  to  approaches 
used  in  commercial  cases.  The  study  should  identify  particular  problems  and 
needs  related  to  applying  maintenance  data  collection  transparency  to  DoD 
systems. 

5.2  LONG  TERM  ACTIONS 

Based  on  the  results  of  the  near  term  actions  and  in  order  for  DoD  to  realize  the 
full  potential  of  the  integrated  diagnostics  concept,  there  are  several  implementation  and 
environment  issues  that  relate  to  the  uniqueness  of  the  DoD  type  missions  that  must  be 
addressed. 


a.  Several  of  the  commercial  cases  demonstrated  that  there  was  benefit  in  using 
data  that  was  rapidly  fed-back  from  maintenance  to  influence  design  and 
system  evolution.  As  a  result,  these  commercial  enterprises  developed  a  link 
to  accomplish  this  feed-back  rapidly.  DoD  should  assess  the  usefulness  of  this 
idea  and  assess  how  DoD  might  go  about  implementing  such  a  concept. 

b.  Since  DoD  has  its  own  missions  and  needs  which  are  different  than  those 
found  in  commercial  cases  and  since  DoD  has  a  large  and  varied  scale  of 


18 


maintenance  activities  to  manage,  DoD  should  develop  a  planning  framework 
to  facilitate  the  implementation  of  integrated  diagnostics  concepts. 

5.3  GENERAL  WORKSHOP  CENSENSUS 

a.  The  workshop  participants  expressed  concern  that  90  percent  of  the  DoD 
systems  in  the  field  today  would  still  be  operational  (and  in  critical  deterrent 
roles)  in  20  years  and  observed  that  opportunities  to  improve  diagnostic 
performance  of  existing  DoD  systems  abound.  Their  concern  was  further 
exacerbated  by  the  perception  that  the  user  communities  are  not  cognizant  of 
the  opportunities  and  benefits  associated  with  integrated  diagnostics. 

Therefore,  based  on  the  results  of  the  workshop,  the  participants  concluded 
that  the  integration  of  diagnostic  elements  provides  a  unique  opportunity  for 
DoD  to  significantly  enhance  the  maintenance  and  availability  of  today’s 
complex  weapon  systems  and  unanimously  recommended  that  DoD 
undertake  initiatives  to  demonstrate  and  communicate  the  opportunities  in 
integrated  diagnostics. 

(1)  DoD  should  accomplish  demonstrations  on  fielded  weapon  systems  to 
further  evaluate  various  integrated  diagnostics  concept  payoffs  and  to 
better  understand  the  technology  applications  most  useful  for  improving 
maintenance. 

(2)  DoD  should  expand  its  efforts  to  increase  the  awareness  and 
understanding  of  integrated  diagnostics  opportunities,  and  document 
demonstrated  benefits  by  presenting  information  to  senior  level  using  and 
maintenance  communities. 
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LIST  OF  ACRONYMS 


AI 

ASR 

BIT 

CAD 

CAE 

CALS 

CAMS 

CSED 

DLH 

DoD 

ED/FI 

EMS 

ESS 

FAA 

FMEA 

FRU 

GEM 

GFE 


Artificial  Intelligence 

Air  Surveillance  Radar 

Built-In-Test 

Computer  Aided  Design 

Computer  Aided  Engineering 

Computer  Aided  Acquisition  and  Logistics  Support 

Computerized  Automative  Maintenance  System 

Computer  and  Software  Engineering  Division 

Lufthansa  German  Airlines 

Department  of  Defense 

Error  Detect,  Fault  Isolate 

Engine  Monitoring  System 

Electronic  Switching  System 

Federal  Aviation  Administration 

Failure,  Mode,  and  Effects  Analyses 

Field  Replaceable  Unit 

Ground-based  Engine  Monitoring 

Government  Furnished  Equipment 

General  Motors 


GM 


IDA 


KLM 

LRU 

LSA 

LSI 

LSSD 

MAM 

NSIA 

OASD(P&L) 

OEM 

R&D 

SAS 

SERD 

SOW 

TCM 

WSIG 

XMAN 


Institute  for  Defense  Analyses 

Royal  Dutch  Airlines 

Line  Replaceable  Unit  ’ 

Logistics  Support  Analysis 

Large  Scale  Integration 

Level  Shift  Scan  Design 

Maintenance  Assist  Module 

National  Security  Industrial  Association 

Office  of  the  Assistant  Secretary  of  Defense  for 
Production  and  Logistics 

Original  Equipment  Manufacture 

Research  and  Development 

Scandinavian  Air  System 

Support  Equipment  Requirements  Document 

Statement  of  Work 

Thermal  Conduction  Module 

Weapon  Support  Improvement  Group 

Expert  Maintenance 
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APPENDIX  A 


WORKSHOP  PARTICIPANTS  &  INVITED  SPEAKERS 


Integrated  Diagnostic  Workshop  Attendees 
21-22  June  1989 


Mr.  Charlie  Alfred 

Fellow  Engineer 

Technical  Services  Department 
Westinghouse  Electric  Corporation 
Integrated  Logistics  Support  Division 

7121  Standard  Dr.,  MS  6060 

Hanover,  MD  21076 

(301)  796-6033 

Mr.  Dick  Allen 

General  Dynamics 

PO  Box  748,  Mail  Zone  4040 

Fort  Worth,  TX  96101 

(817)  935-3244 
(S)  Cindy 

CAPT  Dave  L.  Bailey 

Division  Director  Air  410 

Naval  Air  Systems  Command 

Washington,  DC  20361 

(202)  692-0044 
AV  692-0045 

Mr.  Jon  Beresia 

Director,  Product  Engineering  of  the 
Service  Technology  Group 

General  Motors  Corporation 

Romulus  Engineering  Center 

MS  M-70 

37350  Ecorse  Rd. 

Romulus,  MI  48174 

(313)  595-5900 
Fax  595-5909 
(S)  Lee  Ann 
(313)  595-5900 
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Mr.  David  Blair 

Manager,  Advanced  Development 

Support  Systems 

Westinghouse  Electric  Corporation 

111  Schilling  Rd.,  MS  7034 

Hunt  Valley,  MD  21031 

(301)  584-3518 
Fax  584-3634 
(S)  Donna 

Mr.  Paul  Boozman 

Manager,  EDS/Product  Engineering 

Service  Technology  Group 

General  Motors  Corporation 

Romulus  Engineering  Center 

MS  M-70 

37350  Ecorse  Rd. 

Romulus,  MI  48174 

(313)  595-7540 
Fax  595-5909 
(S)  Carol 
(313)  595-7540 

Mr.  Herbert  R.  Brown 

Research  Staff  Member 

Institute  for  Defense  Analyses 

Computer  and  Software  Engineering  Division 
1801 N.  Beauregard  St. 

Alexandria,  VA  22311 

(703)  845-3521 
Fax  820-9680 

Mr.  Jim  Cigler 

IDSS  Program  Manager 

Naval  Sea  Systems  Command 

Code  CEL-DS4 

Washington,  DC  20362*5101 

(202)  692-0170 
AV  746-3612 

Dr.  Ron  DeHoff 

Manager  of  Maintenance  &  Logistics 

Systems  Control  Technology 

2300  Geng  Rd. 

Palo  Alto,  CA  94303 

(415)  494-2233 

Mr.  Lewis  E.  Dimler 

Research  Staff  Member 

Institute  for  Defense  Analyses 

Computer  and  Software  Engineering  Division 
1801  N.  Beauregard  St. 

Alexandria,  VA  22311 

(703)  845-3507 
Fax  820-9680 
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Mr.  David  L.  Doel 


Mr.  Stephen  P.  Dorey 


Mr.  D.  Kent  Eshelman 


Ms.  Christine  Fisher 


0  Mr.  Jeff  Frenster 


Ms.  Linda  Ginzer 


Mr.  John  Grobler 


Manager  of  Data  Analysis  Technology 
GE  Aircraft  Engineers 
1  Neumann  Way,  Mail  Drop  A321 
Evendale,  OH  45215-6301 

General  Engineer 

Directorate  for  Maintenance 

Maintenance  Engineering  Division 

AMSAV-MEM 

4800  Goodfellow  Blvd 

St.  Louis,  MO  63120 

NAVAIRSYSCOM  Air  41062 
1225  Jefferson  Davis  Hwy 
Crystal  Gateway  2,  Room  413 
Arlington,  VA  22202 

Support  Systems  and  Technology 
The  Pentagon,  Room  2B322 
Washington,  DC  20301-8000 

Engineer  for  XMAN 
Systems  Control  Technology 
2300  Geng  Rd. 

Palo  Alto,  CA  94303 

Department  Head,  Special  Systems 
Development  Department 
AT&T  Bell  Labs 
2000  N.  Naperville  Rd. 

Naperville,  IL  60566 

In-Service  Engineering  Dept.  Head 

NUWES  Bldg.  5092 

Keyport,  Washington  98345-0580 


(513)  583-4832 
Fax  583-4857 
(S)  Phyllis 
(513)583-4261 

(314)  263-3300 


AV693-3300 


(202)  692-4915 


(202)  756-8420 


(415)  494-2233 
Ext  520 


(312)  979-6584 
Fax  979-6745 
(S)  Gale  Taborsky 
312-979-6583 

(206)  396-4200 
AV  744-4200 
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Mrs.  Debra  Haley 


CAPT  John  Hall 


LTC  Charles  Holzler 


Mr.  Anthony  Wayne  Horton 


Mr.  Steve  Hutchings 


Mr.  William  M.  Kasper 


Chief  Scientist,  Logistics  Research 
ALD/CCN 

Wright-Patterson  AFB,  OH  45433 
Director 

Logistics  and  Maintenance  Policy  Division 
Code  AIR-411 

Naval  Air  Systems  Command 
Washington,  DC  20361-4100 

Department  of  the  Army  Logistics 
Headquarters  Department  of  the  Army 
Office  of  the  Chief  of  Staff  for  Logistics 
ATTN:  DALO-SMC 
Washington,  DC  20310 

Acting  Director 

Maintenance  Engineering  Directorate 
Commanding  General 
US  Army  Missile  Command 
AMSMI-LC-ME(  Anthony  Horton) 

Bldg  5681,  Room  A2C 
Redstone  Arsenal,  AL  35898-5230 A 

LHX  Super  Team 
Bldg  530,  Mail  Stop  B226 
5000  East  McDowell  Rd. 

Mesa,  Arizona  85202 

Deputy  Director  of  Combat 
Development 

US  Army  Ordnance  Center  &  School 
ATTN:  ATSL-CD 

Aberdeen  Proving  Ground,  MD  21005-5201 


(513)  255-4028 


692-3212 


(202)  694-3227 
ext  8553 

AV  227-3227/8553 


(205)  876-4223 
AV  746-4223 


(602)  891-4633 


(301)  671-5698 
ext  5375 

AV  298-5698/5375 


Mr.  Taira  Kato 


Mr.  Jim  Larsen 


Integrated  Diagnostics 
Lockheed 

Dept.  6643  Bldg  65  Plant  A1 
POB  551 

Burbank,  CA  91520 

Division  Chief,  Training  Labs 
HQ  TRADOC 
ATTN:  ATTG-CT 
Fort  Monroe,  VA  22651-5265 


Mr.  Clayton  (Dick)  R.  Lee  Director  of  Material  Systems  & 

Technical  Advisor  to  the  Commander 
US  Army  Logistics  Center  and  Fort  Lee 
ATTN:  ATCL-M 
Fort  Lee,  Virginia  23801-6000 

Mr.  Jack  Lunden  Vice  President 

Orlando  Operations: 

Harris  Corporation  GSSD 
12249  Science  Drive 
Orlando,  FL  32826 


Mr.  Robert  G.  Mager 


Mr.  Jim  Mahanna 


Mr.  Ernie  Martino 


Director  of  Aircraft  Support  Engineering 
Mail  Code  2212042,  PO  Box  516 
St.  Louis,  Missouri  63166 

Intergated  Diagnostic  Manager 
Sikorsky  Aircraft 
N.  Main  St. 

Stratford,  CN  06601 

Vice  President  of  Programs 
c/o  Mr.  Lunden’s  Office 
Harris  Corporation 
12249  Science  Drive 
Orlando,  FL  32826 


(818)  847-4035 


(804)  727-3607 
AV  680-3607 


(804)  734-3271 
ext  3418 
687-3271/3413 


(407)  380-3700 
x4930  x4936 
(S)  Cheryl 


(314)  232-0927 


(203)  383-3481 


(516)  677-2628 


27 


Lt  Col  Thomas  (Tom)  May 


Mr.  Martin  A.  Meth 


Mr.  Larry  Miller 


Mr.  Joe  Mills 


Dr.  Ron  Morris 


CAPT  David  P.  Mozgala 


Dr.  Robert  Osborne 


Mr.  Ed  Peretich 


ATF  DPML 
ASD/YFL 

Wright-Patterson  AFB,  OH  45433 

Director  OASD  ( A/L)  WSIG 
The  Pentagon,  Room  2B322 
Washington,  DC  20301-8000 

Systems  Control  Technology 
2300  Geng  Rd. 

Palo  Alto,  CA  94303 

US  Army  Aviation  Logistics  School 
ATTN:  ATSQ-LCD-M 
Fort  Eustis,  VA  23604-5414 

Manager  of  Product  Assurance 
1  Northrop  Ave. 

Hawthorn,  C  A  90250 

NAVAIR  CASS  PM 
PMA  260 

Washington  DC,  20361 

Manager,  Diagnostics  and 
Monitoring  Department 
Generation  Technology  Systems  Division 
The  Quadrangle 
4400  Alafaya  Trail 
Orlando,  FL  32826-2399 

Advanced  Support  Branch  Head 
Department  of  the  Navy 
Naval  Air  Systems  Command 
ATTN:  AIR-4112 
Washington,  DC  20361-4112 


(513)  255-5785 
AV  785-5785 

(202)  697-0051 
Fax  697-3362 


(415)  496-6529 


(804)  878-6847 
AV  927-6847 

(213)  332-8050 


(202)  692-3581 
AV  222-3581 


(407)  281-3230 
Fax  281-2080 


(703)  692-2445 
(S)  Kathy  Moore 
(703)  692-7071 
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Ms.  Betty  S.  Pinna 


Mr.  Blair  Poteate 


COL  Richard  J.  Raymont 


Dr.  Robert  M.  Rolfe 


Col  Ed  Rooney 


Mr.  Victor  A.  Scuderi 


Mr.  Mike  Simpson 


Secretary 

Institute  for  Defense  Analyses 

Computer  and  Software  Engineering  Division 

1801  N.  Beauregard  St. 

Alexandria,  VA  22311 

AATD  (AVSOM) 

ATTN:  SAVRT-TY-ASR 
Fort  Eustis,  VA  23604-5777 

Director  of  Combat  Development 
US  Army  Ordnance  Missile  &  Munitions 
Center  &  School 
ATTN:  ATSK-CO 
Redstone  Arsenal,  AL  35897-6000 

Research  Staff  Member 
Institute  for  Defense  Analyses 
Computer  and  Software  Engineering  Division 
1801  N.  Beauregard  St. 

Alexandria,  VA  22311 

Deputy  Commander  Maintenance 
325th  Tactical  Training  Wing 
Tyndall  AFB,FL  23401 

Manager  of  Department  W33A 
IBM  -  Bldg  400/042 
9500  Godwin  Dr. 

Manassas,  VA  22110 

Alternate  for  Col  Bullock 
AMC-PM-TMDE 
Bldg  910 

Fort  Monmouth,  NJ  07703 


(703)  824-5532 
Fax  820-9680 


(804)  878-2822 
Fax  878-5058 

(205)  876-3508 
AV  746-3508 


(703)  845-3428 
Fax  820-9680 


AV  523-4214/1 5 


(703)  367-2348 
(S)  Tracy  Hunt 


(201)  532-1301 
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Dr.  Nandakumar  N.  Tendolkar  IBM 

Dept  D18,  Bldg  207 


Poughkeepsie,  NY  12602 


Maj  A.S.  Torigian 


Mr.  Dave  Troskey 


CDR  Ron  Wagner 


COL  Thomas  M.  Walker 


CDR  Denny  Williams 


Mr.  Pete  H.  Williamson 


Dr.  Leo  Young 


GIMADS  Program  Manager 
Director  of  Support  Systems  Directorate 
ASD/ENE  (GIMADS) 

Wright-Patterson  AFB,  OH  45433-6503 

LHX  Prog.  Mngr.  Office 
AMC  PEO-LHX-Q 
4300  Goodfellow 
St.  Louis,  MI  63120-1798 


(513)  255-2509 
AV  785-2509 
(S)  Mary  Lou  Barry 


(314)263-1855 
AV  263-1855 
(S)  Nicky 


COMNAVAF  (804)  444-2541 

Atlanta  Fleet,  Code  533 
Norfolk,  VA  23511 


Assistant  Commandant 

US  Army  Aviation  Logistics  School 

ATTN:  ATSQ-AC 

Fort  Eustis,  VA  23604-5414 

OP  514G 

Department  of  Navy 

Office  of  Chief  Naval  Operations 

Washington,  VA  2350-2000 

Director  Automatic  Test  Systems 
General  Dynamics  Division 
P.O.  Box  85227 
San  Diego,  CA  92138 

Engineering  Technology  Division 
DDDRE  Research  &  Advanced  Technology 
The  Pentagon,  3D1089 
Washington,  DC  20301-3080 


(804)  878-6847 
AV  927-6847 
(S)  Gail 

(703)  697-5448 
Fax  694-5965 


(619)  573-7529 
Fax  573-2266 


(703)  695-0552 
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OPPORTUNITIES  IN  INTEGRATED  DIAGNOSTICS  PRESENTATION  CHARTS 
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MARTIN  A.  METH 

OFFICE  OF  THE  SECRETARY  OF  DEFENSE 
U.  S.  DEPARTMENT  OF  DEFENSE 
JUNE  21-22, 1989 
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LONG  -  TERM  STRATEGY 
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SIGNIFICANT  REDUCTION  IN  LOGISTICS  DEPLOYMENT 
TAIL 


LONG  -  TERM  STRATEGY  (CONTINUED) 
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HOLD  WORKSHOP  TO  BUILD  INDUSTRY  AND  USER  CONSENSUS  FOR 
NEW  MAINTENANCE  "PACKAGE"  APPROACH 


CONFIGURATION 


OPPORTUNITIES 


IMPROVED  FAULT  TREE  LOGIC 


WORKSHOP  PARTICIPANTS 
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APPENDIX  C 


SUMMARIES  OF  CASE  STUDIES 

The  following  sections  contain  summaries  of  each  of  the  case  history  presentations  as  well 
as  a  listing  of  system  attributes  and  general  factors  that  influenced  the  diagnostic 
successes  described  in  the  respective  case  study.  The  case  studies  are  documented  in  the 
order  they  were  presented  in  the  workshop. 
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1.  PRESENTATION  TITLE:  THE  GENERAL  MOTORS  (GM) 
COMPUTERIZED  AUTOMATIYE  MAINTENANCE  SYSTEM 
(CAMS) 

1.1  Summary 

Mr.  Jon  Beresia,  Director,  Product  Engineering  of  the  Service  Technology  Group, 
General  Motors  Corporation,  described  the  GM-CAMS  system  as  an  advanced  vehicle 
service  system.  It  provides  a  nationwide  connection  of  terminals,  networks  and  GM 
factory  computers.  It  also  provides  dealer  service  departments  with  similar  test  and 
diagnostic  capabilities  used  in  assembly  plants,  and  is  designed  with  diagnostic 
capabilities  for  electronic  system  and  powertrain  drivability  in  late  model  GM  vehicles. 


Mr.  Beresia  highlighted  the  combined  market  and  technology  forces  which  led  to 
the  development  of  the  GM-CAMS.  In  general,  the  recent  market  forces  in  the 
automobile  industry  are  being  strongly  influenced  by  the  perceived  maintenance  and 
supportability  of  the  vehicle.  Yet  at  this  same  time  two  other  technology  driven  forces  are 
active  making  the  service  mechanic’s  job  more  difficult.  The  more  restrictive  emission 
controls  and  the  application  of  microcomputer-based  control  systems  in  the  engines  and 
transmissions  are  making  the  vehicles  more  complex. 

The  technology  in  the  complex  electrical  and  mechanical  systems  of  today’s 
vehicle  are  advancing  faster  than  the  mechanic’s  expertise  and  training.  This  condition 
was  exemplified  by  the  fact  that  frequency  of  “trouble  not  found”  problems  on  the 
vehicles  with  the  more  advanced  technology  were  increasing  significantly.  Furthermore, 
the  technique  of  troul  le-shooting  by  removing  and  replacing  suspected  parts  was 
becoming  the  accepted  practice  and  driving  up  the  customer’s  repair  costs  as  well  as 
impacting  the  parts  warranty  service  costs  for  GM.  GM  recognized  that  a  new  diagnostic 
capability  was  needed  and  developed  the  GM-CAMS  expert  (rule-based)  system. 


The  ability  to  service  the  vehicles  is  a  function  of  many  factors  including  the 
maintenance  documentation,  the  training  of  the  service  technician,  the  test  and 
diagnostic  equipment,  and  the  ability  to  update  and  maintain  the  service  bulletins, 
campaigns  (block  design  changes),  and  the  various  specifications  for  each  type 
subsystem.  Mr.  Bereisa  explained  that  the  GM-CAMS  system  evolved  into  a  technician’s 
terminal  that  is  much  more  than  a  powerful  diagnostic  computer — it  is  an  integrated 
diagnostic  tool  that  provides  the  following  vehicle  service  capabilities: 


a.  Provides  up-to-date,  relevant  service  information,  such  as  service  bulletins, 
campaigns  and  vehicle  specifications  all  of  which  are  easily  called  up  and 
viewed  on  the  monitor. 
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b.  Provides  in-depth  display  and  analysis  of  out-of-range  vehicle  data  streams 
and  malfunction  codes. 

c.  Performs  a  quick  test  of  entire  vehicle  electronic  systems  under  key-on,  part 
throttle  and  idle  conditions. 

d.  Performs  in-depth  diagnosis  and  fault  isolation  of  vehicle  electronic  systems 
and  components  down  to  harness  wires  and  connector  pins  without  the  need  of 
additional  tools  or  “trouble  tree”  charts. 

e.  Supports  difficult-to-make  adjustments,  such  as  minimum  air  rate  or  throttle 
position  sensor,  without  the  need  for  specifications  or  additional  tools. 

f.  Transmits  vehicle  data  back  to  GM  divisional  technical  assistance  centers  for 
simultaneous  viewing  of  problem  vehicle  performance.1 

Mr.  Bereisa  observed  that  the  GM-CAMS,  with  the  centralized  data  collection 
and  analyses  capabilities,  is  providing  an  effective  bridge  between  the  maintenance 
technician  and  the  system  designer.  The  maintenance  data  feedback  is  essential  for 
maturing  both  the  system  design  as  well  as  the  diagnostics.  Mr.  Bereisa  also  indicated 
that  GM  is  attempting  to  further  extend  this  bridge  by  maintaining  a  single  thread  of 
diagnostic  requirements  that  are  needed  throughout  the  design,  preservation  and 
maintenance  life  cycle. 

Table  C-l  summarizes  the  GM-CAMS  diagnostics  system  attributes  and  factors 
that  were  assessed  to  influence  the  -rogram  success. 


1.  “GM-CAMS.  The  Technical  Connection,”  Information  pamphlet  (M12727),  EDS,  Dec  87. 


Table  C-l.  GM-CAMS  Diagnostics  System 


•  Addresses  a  clearly  defined  warranty  parts  problem. 

•  Problem  resolution  had  corporate  level  support. 

•  Diagnostic  system  design  and  evolution  focused  on  the  specified  problem. 

•  Set  out  to  build  an  integrated  set  of  diagnostics  elements. 

•  Applies  existing  technology  components. 

•  Automates  configuration  management  of  released  product  and  support  systems. 

•  Uses  central  data  collection  (including  real  time  measurement  data). 

•  Uses  central  analysis/refinement  to  mature  diagnostics  and  product. 

•  System  significantly  improves  fault  isolation  and  diagnostics  accuracy. 

•  Uses  functional  condition  sensors. 

•  Takes  advantage  of  functional  sensor  information  overlap  to  improve  diagnostics 
and  system  reliability. 

•  Uses  automatic  detection  and  error  logging  of  intermittent  and  hard  failures 
(transparent  to  operator). 

•  Makes  use  of  automated  (on-equipment)  operational  fault  log. 

•  Uses  diagnostics  that  self-adapt  to  vehicle  condition  and  environment. 

•  Delivers  intelligence  presentation  of  maintenance  information  electronically. 

•  Simplifies  maintenance  input  from  technician. 


•  Uses  software  to  compensate  for  limitations  of  hardware  BIT  extensively. 
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2.  PRESENTATION  TITLE:  IBM  3090  DIAGNOSTIC  SYSTEM 

2.1  Summary 

Dr.  Nandakumar  Tendolkar,  Senior  Engineer,  Noise  Detection  and  Recovery 
Design  Department,  IBM  -  Data  Systems  Division  presented  the  IBM  3090  Diagnostic 
System.  The  IBM  3090  is  the  latest  generation  general  purpose  System/370  computer. 
The  3090  Processor  includes  a  number  of  units:  Processor,  Processor  Controller,  a 
Coolant  Distribution,  and  a  Power  Unit.  The  Processor  Controller  is  an  independent 
processor  dedicated  to  maintenance  and  monitoring  functions. 

The  processor  is  made  from  a  very  unique  packaging  technology  called  the 
Thermal  Conduction  Module  (TCM),  which  is  comprised  of  helium  filled  multi-chip  large 
scale  integration  (LSI)  modules.  Each  TCM  contains  132  silicon  chips.  The  next  level  of 
packaging  is  based  on  six  to  nine  TCM’s  per  board.  On-line  fault  isolation  provides  for 
fault  isolation  to  one  field  replaceable  unit  (or  TCM  board)  in  95%  of  maintenance 
actions. 

The  maintenance  strategy  at  the  customers  site  is  based  upon  a  diagnostic  system 
which  fault  isolates  to  the  defective  TCM  board.  The  maintainer  can  then  replace  the 
identified  board.  Intermittent  as  well  as  hard  faults  are  expected  as  fault  types  that  the 
diagnostic  system  must  detect  and  identify.  The  basic  strategy  of  fault  detection/isolation 
has  been  moved  from  a  separate  diagnostic  process  which  may  never  determine  or 
reproduce  an  intermittent  fault  to  a  concurrent  detection  and  identification  strategy  based 
upon  hardware  error  detection,  error  logging,  and  fault  isolation  by  analysis  of  logged 
data.  IBM  calls  this  approach  ED/FI  (error  detect,  fault  isolate).  The  Processor 
Controller  manages  the  process  of  identifying  the  failed  field  replaceable  unit  (FRU). 
The  customer  engineer  will  change  the  FRU  and  the  Processor  Controller  is  used  to  verify 
the  repair. 

The  maintenance  processor  (the  Processor  Controller)  provides  the  mechanism 
for  monitoring  and  supervising  error  recovery  and  fault  isolation.  The  basic  hardware  is 
designed  with  Level  Shift  Scan  Design  (LSSD)  built-in-test  (BIT).  The  maintenance 
processor  has  complete  control  and  access  to  this  BIT  and  manages  fault  recovery  as  well 
as  diagnostic  actions  required  to  verify  the  problem  FRU.  A  history  of  all  faults  is 
maintained  and  when  the  processor  detects  an  error  that  exceeds  a  designated  threshold, 


the  maintenance  processor  notifies  the  customer  engineer  which  FRU  to  replace.  The 
basic  maintenance  approach  is  a  two-level  maintenance  strategy. 

The  maintenance  processor  uses  a  fault  isolation  analysis  strategy  based  on 
intersection  of  direct  error  logging  and  fault  dependency  analysis  to  determine  the  failed 
FRU. 

IBM  set  out  on  a  new  strategy  to  eliminate  downtime  for  diagnostic  testing.  This 
new  concurrent  error  detecting,  and  fault  isolating  strategy  has  allowed  IBM  to  provide 
high  system  availability  with  a  two  level  maintenance  approach. 

Table  C-2  summarizes  the  IBM  3090  system  attributes  and  factors  that  influenced 
case  study  success. 

Table  C-2.  IBM  3090  System 

•  The  3090  system  is  a  primary  product  of  IBM. 

•  Willing  to  dedicate  significant  system  resources  to  address  diagnostic 
requirements  (approximately  1/4  of  system  hardware). 

•  Conducted  extensive  design  and  testability  analysis  before  committing  to 
hardware  configuration. 

•  Uses  software  to  compensate  for  limitations  of  hardware  BIT  extensively. 

•  Leading  edge  technology  selected  and/or  developed  as  needed  to  meet  diagnostic 
objectives. 

•  Automatic  detection,  error  logging,  and  isolation  of  intermittent  and  hard  faults 
established  as  primary  goal  at  design  onset. 

•  Established  a  two-level  maintenance  goal  with  parts/cards  fault  isolation, 
removal  and  replacement  objective. 

•  Made  extensive  use  of  diagnostics  analysis  tools. 

•  Used  diagnostic  software  to  identify  and  isolate  intermittent  faults. 

•  Used  checking  algorithms  (implemented  in  software)  to  allow  reconfiguration 
around  defective  redundant  elements. 


3.  PRESENTATION  TITLE:  AT&T  SWITCHING  SYSTEMS  USE  OF 
INTEGRATED  MAINTENANCE  AND  DIAGNOSTICS 

3.1  Summary 

Ms.  Linda  Ginzer,  Head,  Special  Systems  Development  Department,  AT&T  Bell 
Laboratories,  presented  the  AT&T  #5ESS  integrated  diagnostic  system.  The  #5ESS  is 
the  latest  generation  digital  switching  system  developed  and  manufactured  by  AT&T.  It 
provides  local  and  toll  switching  as  well  as  operator  services  to  the  telephone  network. 
The  switch  supports  100,000  line  terminations,  400,000  calls  per  hour,  and  the  installed 
base  of  #5ESS  is  twenty-three  million  lines.  The  system  has  very  high  reliability 
requirements  with  a  system  downtime  of  less  than  three  minutes  per  year,  and  a  single  line 
downtime  of  less  than  twenty  eight  minutes  per  year.  The  system  availability  is  99.8%. 
Downtime  in  such  a  high  reliability  system  based  on  a  fault  tolerant  system  is  20%  due  to 
hardware,  40%  due  to  embedded  software,  and  40%  due  to  maintenance  technician 
procedural  mistakes. 

The  #5ESS  does  not  have  a  conventional  electronic  computer  system 
maintenance  philosophy  where  errors  are  catastrophic,  and  system  downtime  is  nuisance. 
Instead  system  downtime  is  a  catastrophe,  and  errors  become  a  nuisance.  The  system  is 
based  on  distributed  hardware,  localized  fault  detection  and  recovery,  and  fault 
containment  from  the  balance  of  the  system.  The  system  uses  continuous  self-testing  of 
all  hardware  and  software,  and  supports  graceful  degradation  of  system  functions  based 
on  hardware  redundancy.  The  system  includes  error  reporting  and  on-line  trouble 
location  reporting.  Generally,  the  system  corrects  faults  automatically  and  notifies  the 
craft  (maintainer)  of  circuit  packs  to  be  replaced.  Software  error  checks  are  based  on 
correlation  of  redundant  information  and  autonomous  process  checks  as  well  as  more 
typical  data  format  and  range  checks. 

The  diagnostic  system  supports  craft  control  of  diagnostic  processes  to  verify 
operation  of  a  circuit,  confirm  a  real-time  fault  recovery  :  ecision,  and  is  also  used  during 
manufacture  and  installation  of  the  system. 

The  craft  maintenance  interface  actually  identifies  which  circuit  pack  is  suspected 
as  faulty  indicating  aisle,  frame,  bay  location  as  well  as  identification  codes.  Ninety 
percent  of  the  diagnostic  applications  identify  one  circuit  pack,  and  the  balance  are 
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resolved  to  two  circuit  packs.  Circuit  packs  are  returned  to  the  factory  for  conventional 
repair  by  replacement  of  faulty  components. 


This  level  of  required  system  availability  dictated  that  approximately  fifty 
percent  of  the  software  be  dedicated  to  maintenance  and  diagnostics.  Much  of  the 
diagnostic  software  makes  up  for  limitations  in  the  capability  of  built-in-test  hardware.  A 
significant  percentage  of  the  operating  system  software  is  used  to  support  real-time  error 
detection  and  management.  The  #5ESS  provides  a  guidepost  to  attainable  availability 
goals  for  electronic  systems  and  demonstrates  effective  two  level  maintenance. 

Table  C-3  summaries  the  5ESS  system  attributes  and  factors  that  influenced  case 
study  success. 


Table  C-3.  5ESS  System 


•  System  availability  of  99.8%  (or  3  minute  down  time  per  year)  is  primary 
corporate  objective. 

•  Different  maintenance  philosophy  errors  are  a  nuisance;  however,  a  system 
failure  represents  a  catastrophe. 

•  Predefined  maintenance  strategy  to  provide  redundancy  and  prevent  down  time. 

•  Automatic  detection,  error  logging,  and  isolation  of  intermittent  and  hard  failures 
are  a  design  requirement. 

•  Diagnostic  elements  implementation  applied  leading  edge  technology  solutions 
when  required. 

•  The  system  automates  maintenance  and  resource  allocation  in  real  time  without 
an  operator. 

•  Diagnostic  and  fault  recovery  are  built  into  the  design. 

•  Simplified  maintenance  technician  interface. 

•  The  system  uses  extensive  software  to  compensate  for  limitations  of  hardware 
BIT  coverage  and  intelligence. 


4.  PRESENTATION  TITLE:  WESTINGHOUSE  TURBINE 
GENERATOR  ON-LINE  DIAGNOSTIC  SYSTEM 

4.1  Summary 

Dr.  Robert  Osborne,  Manager,  Diagnostic  and  Monitor  Development, 
Generation  Technology  Systems  Division,  Westinghouse,  described  the  on-line 
diagnostics  for  commercial  turbines,  generators  and  steam  chemistry  systems.  The 
Westinghouse  system  applies  artificial  intelligence  (AI),  rule-based  logic  to  provide 
continuous  on-line  status  and  problem  diagnostics  information  (including  the  projection  of 
potential  future  problems)  to  a  single  centralized  control  center. 

Dr.  Osborne  explained  that  the  downtime  of  a  major  unit  can  cost  as  much  as  a 
half  million  dollars  a  day.  The  AI  diagnostics  permit  utilities  using  these  systems  to  plan 
maintenance  intervals  with  higher  levels  of  confidence  and  to  plan  for  the  higher  levels  of 
availability.  The  diagnostic  systems  are  continually  fed  information  from  on-line  sensors 
that  monitor  the  system  operating  conditions.  The  operating  parameters  and  other  data 
are  then  transmitted  to  the  diagnostic  center’s  centralized  computers. 

The  computers  are  programmed  to  simulate  the  thinking  process  of  the  system 
experts  and,  based  on  the  extensive  rule-based  logic  and  the  real  time  operating  data,  to 
recommend  appropriate  actions.  In  addition  to  identifying  recommended  actions,  the  AI 
diagnostic  system  identifies  specific  equipment  conditions,  places  conditions  in  a  relative 
priority  order,  and  identifies  potential  consequences  of  inaction. 

Dr.  Osborne  also  indicated  that  because  the  data  and  rules  are  continually 
exercised  at  the  centralized  diagnostic  center  for  multiple  on-line  systems,  the  diagnostic 
rule  based  for  any  given  situation  grows  faster  with  the  experience  that  may  be  gained 
from  a  greater  number  of  systems.  He  indicated  that  the  bottom  line  is  increased 
productivity  because  maintenance  actions  can  be  scheduled  on  a  convenience  basis  rather 
than  due  to  reactive  necessity.  Based  on  the  estimated  half  million  dollars  per  day  for  a 
major  system  outage  and  the  on-line  availability  improvement  of  approximately  1.9 
percent  (or  7  days),  they  generally  estimate  the  economic  return  to  be  3.5  million  dollars 
annually  per  application  of  the  AI  diagnostics  system. 

Another  interesting  aspect  of  this  AI  diagnostic  concept  is  that  virtually  all  of  the 
installations  were  applied  to  existing  turbine  and  generator  systems.  Not  only  does  it 


represent  an  application  of  the  leading  edge  technology,  it  represents  a  stand-alone 
innovation  to  an  existing  system  that  will  provide  opportunities  for  better  maintenance 
management  and  planning  as  well  as  increased  productivity. 


Table  C-4  summarizes  the  GenAid  diagnostic  system  attributes  and  factors  that 
influenced  case  study  success. 

Table  C-4.  GenAid  Diagnostic  System 


•  Corporate  need  to  minimize  unscheduled  downtime  (cost  approximately  $.5M 
per  day). 

•  Establishes  an  environment  that  permits  continuing  maturation  of  diagnostics 
(even  with  low  failure  rates  and  a  small  population  of  monitored  units). 

•  Expert  (rule  base)  system  captures  what  has  been/is  being  learned,  thereby 
improving  diagnostic  capability. 

•  Requires  prognostic  view  to  ensure  early  identification  of  necessary  maintenance 
action  as  well  as  consequences  of  inaction. 

•  Requirements  driven  by  users  (implemented  on  existing  system  by  3rd  party). 

•  Uses  remote  central  data  collection,  analyses,  and  AI  maintenance  prognostics. 

•  Central  on-line  analysis  of  multiple  (similar)  systems  improves  diagnostics 
algorithms  and  rules. 

•  Corporate  need  to  minimize  catastrophic  (secondary  damage)  problems. 

•  Uses  functional  and  diagnostic  condition  sensors. 

•  Takes  advantage  of  functional  and  diagnostic  sensor  overlaps  to  improve 
diagnostic  accuracy. 

•  Changes  from  on-condition  maintenance  strategy  to  prognostic  based 
maintenance  planning. 
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5.  PRESENTATION  TITLE:  COMMERCIAL  ENGINE 

INTEGRATED  DIAGNOSTICS  EXPERIENCE  -  THE  GROUND 
BASED  ENGINE  MONITORING  (GEM)  PROGRAM 

5.1  Summary 

Mr.  David  Doel,  Manager  of  Data  Analysis  Technology,  General  Electric 
Aircraft  Engines,  described  their  Ground-based  Engine  Monitoring  (GEM)  system. 
GEM  was  developed  in  the  early  1980s  by  GE  Aircraft  Engines  in  cooperation  with  DLH 
(Lufthansa  German  Airlines),  KLM  (Royal  Dutch  Airlines)  and  SAS  (Scandinavian  Air 
System).  The  original  objectives  of  the  GEM  included  the  following:  (1)  providing  a 
single  integrated  engine  monitoring  software  system,  (2)  reducing  the  personnel 
requirements  for  engine  monitoring  support,  (3)  increasing  the  effectiveness  of  engine 
monitoring  equipment,  (4)  capturing  history  to  facilitate  future  monitoring 
improvements,  (5)  providing  a  single  software  package  to  address  all  engine  models,  and 
(6)  integrating  to  other  airline  systems. 

The  GEM  application  scenario  captures  on  board/in-flight  engine  operation  data 
as  well  as  test  cell  and  maintenance  data.  The  aircraft  data  is  sent  to  a  centralized  real¬ 
time  processing  center  via  the  line  station  reservation  system.  The  ground  based  software 
analyzes  the  engine  performance  data  from  the  test  cell,  run-ups,  and  on-wing.  Analytic 
features  of  the  GEM  include  maintenance  alerts,  the  recognition  of  potential 
performance  or  maintenance  trends,  data  on  the  occurrence  of  exceeded  limits  of 
specified  operating  parameters,  and  summary  data  on  fleet  averages. 

The  GEM  data  and  analyses  provide  early  identification  and  resolution  of  field 
problems,  and  through  the  alerts  and  trend  analyses,  reduces  personnel  needs  for 
monitoring.  The  fan  rotor  imbalance  analysis  alone  is  estimated  to  save  $1000  per  trim 
balance  with  an  average  of  two  trim  runs  per  year  per  engine.  On-wing  maintenance 
performance  analysis  saves  an  estimated  5  percent  of  the  maintenance  overhaul  budget. 
Furthermore,  the  controls  monitoring  function  saves  fuel  and  increases  the  engine  stall 
margin  resulting  in  approximately  a  0.5  percent  reduction  in  fuel  consumption. 

Mr.  Doel  indicated  that  the  development  of  the  GEM  system  was  clearly  a  team 
effort,  and  that  the  development  and  implementation  occurred  in  evolutionary  stages.  He 
indicated  that  the  monitoring  and  analyses  technologies  can  not  develop  by  theory  alone. 


For  example,  new  designs  are  via  development  testing,  and  the  integrated  diagnostics 
that  evolve  must  be  tested  in  service  environments.  This  is,  in  large  part,  due  to  the  fact 
that  simulated  tests  cannot  provide  sufficiently  large  data  samples  that  are  representative 
of  real  operational  conditions.  Finally,  it  takes  time  to  fully  comprehend  what  the  data 
may  be  implying  and  to  establish  a  means  of  integrating  the  results  of  various  data  inputs. 

Mr.  Doel  highlighted  the  following  lessons  learned  from  the  initial  development  of 
the  GEM.  Initial  development  of  monitoring  systems  is  truly  a  research  area  and  it  needs 
the  extensive  testing  data  from  field  experience.  He  indicated  that  deriving  up  front 
system  development  and  production  specifications  for  integrated  diagnostics  capabilities, 
may  be  very  difficult  due  to  the  iterative  nature  of  diagnostics  development.  He  went  on 
to  suggest  that  rapid  prototyping  environments  are  needed  to  support  integrated 
diagnostic  research.  These  prototyping  concepts  should  recognize  that  the  sensor  and 
hardware  designs  are  generally  not  easy  to  alter;  however,  the  software  may  be  designed 
in  such  a  way  that  it  may  be  enhanced  as  field  experience  is  gained. 

Mr.  Doel  also  provided  the  following  observations.  For  integrated  diagnostics  to 
progress,  the  designer  and  user  communities  must  take  some  risks.  He  further  observed 
that  integrated  diagnostics  is  a  test  of  perseverance  and  it  is  often  very  difficult  to  justify 
the  addition  of  specific  sensors  or  monitors  into  a  system  design  to  support  diagnostics. 
They  cost  more  to  add,  they  reduce  reliability,  and  they  can  cause  other  system  design 
and  maintenance  problems.  Mr.  Doel  suggested  that  integrated  diagnostics  must  be  sold 
on  faith  because  very  often  the  available  justification  data  will  not  support  the  real  need. 

Table  C-5  summarizes  the  GEM  system  attributes  and  factors  that  influenced 
case  study  success. 


Table  C-5.  GEM  System 


•  Clear  corporate  desire  to  minimize  unscheduled  downtime. 

•  Establishes  an  environment  that  permits  continuing  maturation  of  diagnostics. 

•  Implements  a  prognostic  capability  to  ensure  early  identification  of  necessary 
maintenance  action  as  well  as  consequences  of  inaction. 

•  Uses  central  data  collection  and  analysis  from  all  on-wing  and  ground 
maintenance  actions  (i.e.,  test  cell,  etc.). 

•  Uses  continuous  centralized  analysis  of  multiple  systems  to  improve  diagnostic 
algorithms. 

•  Corporate  need  to  minimize  catastrophic  (secondary  damage)  problems. 

•  Requirement  driven  by  customers  (integrated  diagnostics  requirement  not  driven 
by  original  equipment  manufacturers  (OEM),  however,  implemented  by  OEM). 

•  Changes  from  on-condition  maintenance  strategy  to  prognostic  based 
maintenance  planning. 

•  Uses  functional  and  diagnostic  condition  sensors. 

•  Takes  advantage  of  functional  and  diagnostic  sensor  overlaps  to  improve 
diagnostic  accuracy. 
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6.  PRESENTATION  TITLE:  THE  SYSTEM  CONTROL 

TECHNOLOGY  XMAN  JET  ENGINE  DIAGNOSTIC  SYSTEM 

6.1  Summary 

Dr.  Ronald  DeHoff,  Manager  of  the  Maintenance  and  Logistics  Systems 
Department,  and  Mr.  Lawrence  Miller,  Manager,  New  Program  Development, 
Maintenance  and  Logistics  Department,  both  of  Systems  Control  Technology,  jointly 
presented  an  overview  of  the  eXerpt  MAiNtenance  (XMAN)  system.  XMAN  is  an 
expert  system  which  has  been  applied  to  integrated  maintenance  support  of  jet  engines  in 
both  Air  Force  and  Navy  weapon  systems.  These  aircraft  engines  are  maintained  at 
three  operating  levels,  and  the  maintenance  process  generally  involves  moving  hardware 
and  data  from  the  flight  line  through  the  overhaul  depot.  The  modern  aircraft  engines 
that  XMAN  is  being  applied  to  have  engine  monitoring  systems  (EMS)  on-board  that 
detect  abnormal  engine  operating  data.  XMAN  supports  the  diagnostics  and 
troubleshooting  by  applying  a  rule  based  integrated  interface  framework  to  the  available 
EMS,  ground  maintenance  and  logistics  data.  XMAN  also  delivers  technical  order 
and/or  manual  data  to  the  engine  mechanic  at  the  flight  line,  thereby  greatly  reducing  the 
paperwork  burden  associated  with  maintenance  management. 

The  main  components  of  XMAN  include  a  computerized  knowledge  or  rule  base, 
integrated  data  bases,  and  a  control  system.  The  data  sources  include  the  EMS  data 
taken  directly  from  the  aircraft,  data  from  both  on-and  off-aircraft  maintenance  actions, 
and  the  base  logistics  maintenance  and  support  information  systems.  This  information  is 
used  to  form  the  historical  records  upon  which  XMAN  applies  the  expert  knowledge. 
This  process  results  in  identification  of  equipment  faults,  provides  prognostic  information 
as  well  as  supporting  data. 

XMAN  is  also  an  automated  technical  manual  delivery  system  that  supplements 
the  go/no  go  diagnostics  and  on-wing  inspections.  It  also  has  time/cycle  tracking  as  well 
as  trends/forecasting  capabilities.  Furthermore,  it  helps  keep  track  of  the  engine  parts 
configuration  and  the  specific  engine  maintenance  history. 

Both  presenters  focused  on  the  challenges  of  applying  an  integrated  diagnostic 
and  expert  system  into  the  DoD  environment.  There  are  a  number  of  training  concerns 
including  computer  literacy  as  well  as  management,  social  and  political  factors  that  must 
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be  considered  in  any  application.  For  example,  they  found  that  XMAN  required  a 
significant  user  and  management  acceptance  period  given  the  potential  safety 
implications  involved  with  a  diagnostic  maintenance  tool  for  aircraft  jet  engines.  The 
apparent  learning  period  necessary  to  accept  the  automated  diagnostics  outputs,  coupled 
with  the  need  to  change  work  habits  and  cultural  barriers,  appears  to  be  consistent  across 
system  applications. 

Table  C-6  summarizes  the  XMAN  diagnostics  system  attributes  and  factors  that 
were  assessed  to  influence  the  program  success. 

Table  C-6.  XM AN  Diagnostics  System 


•  Automates  the  integration  of  existing  maintenance  and  operational  history  data 
bases. 

•  Integrates  the  maintenance  data  base  information  with  the  engine  maintenance 
procedures  and  diagnostics. 

—  Prognostic  (interactive  with  maintenance  technician). 

—  Electronic  presentation  of  technical  manuals  as  needed. 

—  Integrates  electronic  reference  to  technical  manuals. 

—  Provides  support  trend  analyses  and  prognostic-based  maintenance 
actions. 

—  Data  is  centrally  analyzed  and  distributed  for  maintenance 
implementation. 

—  Integrated  training  and  electronic  technical  manual  presentation. 

—  Developed  and  retrofited  to  existing  system  by  third  party  developer. 

—  Provides  maintenance  technology  bridge  between  paper  and  electronic 
presentation. 
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7.  PRESENTATION  TITLE:  THE  WESTINGHOUSE  REMOTE 
MAINTENANCE  SYSTEM  FOR  AIR  SURVEILLANCE  RADAR 
(ASR) 

7.1  Summary 

Mr.  Charles  Alfred,  Fellow  Engineer,  Remote  Maintenance  Systems 
Department,  Electronic  Systems  Group,  Westinghouse,  described  the  new  Federal 
Aviation  Administration  (FAA)  Remote  Maintenance  System  for  Air  Surveillance 
Radars  (ASR).  These  radars,  currently  being  fielded  at  FAA  sites  throughout  the  United 
States,  demousU ale  a  growing  need  and  concern  for  integrated  diagnostics  and  remote 
maintenance  capabilities.  A  primary  driver  for  the  new  FAA  maintenance  philosophy  is 
the  anticipated  budgets  and  funding  cuts  which  will  dictate  that  future  maintenance  and 
support  efforts  must  be  less  costly.  This  condition  is  occurring  at  the  same  time  that  a 
significant  portion  of  the  average  maintenance  workforce  will  be  approaching  retirement 
age,  thus  complicating  the  training  of  a  wide  range  of  technologies  necessary  to  support 
the  old  as  well  as  new  (state  of  the  art)  systems.  Finally,  the  new  systems  will  have  higher 
reliabilities,  thereby,  creating  a  secondary  problem  of  impacting  the  maintenance  on-the- 
job  training  as  well  as  extending  the  time  necessary  to  gain  a  quality  experience  base. 

The  remote  maintenance  systems  for  the  ASR  will  provide  for  monitoring  and 
identification  of  needed  maintenance  actions  from  centralized  sites.  Then,  when  a 
maintenance  problem  is  identified,  a  field  technician  will  be  dispatched  to  perform  the 
required  maintenance  or  support  action.  The  expert  diagnostics  system  will  identify 
needed  parts  to  be  transported  to  the  site  as  well  as  specific  instructions  to  perform  the 
corrective  action.  The  system  will  also  have  the  capability  to  provide  limited  remote 
maintenance  by  switching  to  alternate  redundant  equipment  when  directed  from  the 
central  maintenance  control  facility. 

This  integrated  diagnostics  system  will  also  help  to  improve  the  configuration  and 
control  of  the  FAA  facilities.  It  will  provide  a  common  configuration  and  failure 
information  data  base.  The  centralized  management  and  the  remote  monitoring  will  help 
facilitate  the  standardization  of  both  the  hardware  and  software  documentation  and 
configuration  across  sites. 
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The  remote  maintenance  system  capabilities  remove  the  need  for  on  site/on  call 
personnel,  provide  a  central  control  of  multiple  sites,  and  provide  the  opportunity  for 
sharing  of  on-the-job  maintenance  learning  and  experience. 

In  addition  to  the  basic  system,  Westinghouse  has  developed  a  remote  monitoring 
system  simulator  for  training.  It  provides  the  look  and  feel  of  the  primary  system, 
operational  error  trapping  on-line  help,  as  well  as  a  question  and  answer  capability  via  a 
mail  facility. 


Mr.  Alfred  highlighted  the  following  lessons  learned.  Remote  control  and 
embedded  training  cannot  be  implemented  without  being  initially  specified.  The  remote 
features  and  capabilities  should  be  designed  with  user  input  and  specifically  to  meet  the 
technician’s  needs.  Training  simulations  should  be  developed  along  with  the  primary 
systems,  and  they  need  the  same  configuration  control  attention  to  detail  as  the  primary 
system.  Finally,  user  friendliness  is  essential  and  valuable  inputs  come  from  maintenance 
training  organizations  as  well  as  from  the  maintainers. 

Table  C-7  summarizes  the  ASR  diagnostics  system  attributes  and  factors  that 
were  assessed  to  influence  the  program  success. 
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Table  C*7.  ASR  Diagnostics  System 


•  FAA  corporate  commitment  to  reduce  maintenance  manpower,  spares  and 
support  equipment. 

•  High  system  availability  a  primary  FAA  objective. 

•  Uses  ability  to  take  full  remote  control  of  multiple  sites  from  a  centralized  facility 
to  remotely  fault  isolate,  correct  software  driven  faults,  and  replace  hardware 
failures  with  redundant  equipment. 

•  Eliminates  permanent  on-site  maintenance  staff  through  remote  monitoring, 
diagnostics,  redundancy  and  mobile  field  service  (2-level  maintenance). 

•  Provide  simulation  based  training  (for  remote  maintenance  and  diagnostics). 

•  Reduces  spares  through  improved  diagnostics  accuracy. 

•  Reduces  spares  by  consolidating  inventories  at  centralized  facilities  (control 
centers). 

•  Uses  functional  condition  sensors  and  built-in-test  equipment. 
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8.  PRESENTATION  TITLE:  THE  IBM  AN/BSY-1 

8.1  Summary 

Mr.  Victor  Scuderi,  Project  Manager  of  Integrated  Diagnostics,  IBM-Federal 
Systems,  Systems  Integration  Division  presented  the  AN/BSY-1  Integrated  Diagnostics. 
The  AN/BSY-1,  Submarine  Combat  Control  and  Acoustics  System,  provides  the  688 
class  submarine  with  a  completely  Integrated  Combat  Suite  consisting  of  117  cabinets  of 
hardware  and  4.5  million  lines  of  software.  The  system  relies  heavily  on  previous  sub¬ 
systems. 

The  maintenance  strategy  for  the  submarine  is  based  upon  no  on-board  test, 
measurement,  and  diagnostic  equipment.  On  board  maintenance  is  limited  to  automatic 
reconfiguration  of  fix-on-incident,  and  modules  swaps.  Spare  modules  are  pre-packed  in 
maintenance  assist  module  (MAM)  kits.  The  intermediate  (or  “I”)  level  maintenance 
uses  conventional  maintenance  approaches.  This  system  is  unique  compared  to  other 
high  availability  systems  presented  at  this  workshop.  It  contains  Government  Furnished 
Equipment  (GFE)  and  equipment  produced  by  many  subcontractors.  The  system  on 
board  diagnostic  system  is  based  upon  performance  monitoring  and  fault  localization. 

On-line  performance  monitoring  can  detect  95%  of  faults  in  a  mean  time  of  180 
seconds.  The  maximum  detect  time  is  seven  minutes  and  one  false  alarm  is  allowed  per 
twenty-four  hours.  The  off-line  fault  localization  system  can  isolate  to  two  modules  80% 
of  the  time  and  must  isolate  to  no  more  than  eight  modules. 

Built-in-test  hardware  structural  test  was  available  only  in  the  new  system 
components.  The  system  relied  upon  4.5  lines  of  software  code  to  implement  system 
functions  and  the  maintenance  strategy.  About  25%  of  that  software  was  dedicated  to  the 
maintenance  strategy.  The  AN/BSY-1  reduced  technical  manuals  through  very  effective 
on-board  test  and  fault  isolation  capability.  No  circuit  diagrams  were  required  to  be 
deployed.  On  board  maintenance  manual  space  was  reduced  from  28.5  to  15.7  cubic 
feet.  Circuit  training  was  eliminated  for  the  maintainers. 

The  AN/BSY-1  represents  an  evolutionary  approach  to  maintenance  strategy 
improvement.  Extensive  on-board  software  and  redundant  hardware  reduced  the 
requirements  for  support  equipment  to  maintenance  assist  module  (MAM)  kits  only. 


Table  C-8  summarizes  the  AN/BSY-1  system  attributes  and  factors  that 
influenced  case  study  success. 

Table  C-8.  AN/BSY-1  System 


•  Functional  space  constraints  dictated  reductions  in  paper  manuals,  test 
equipment  and  spares. 

•  Design  and  maintenance  requirement  and/or  strategy  from  the  onset: 

—  No  special  test  equipment. 

—  Reduce  spares  volume. 

•  Essential  customer  requirement  for  high  availability. 

•  Eliminates  requirement  for  circuit  diagrams  on  submarine  through  improved 
diagnostics  accuracy  attained  with  extensive  software. 

•  Uses  checking  algorithms  (implemented  in  software)  to  allow  reconfiguration 
around  defective  redundant  elements. 

•  Uses  software  to  compensate  for  limitations  of  hardware  BIT  extensively. 
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APPENDIX  D 


WORKSHOP  REPORTS 


The  following  sections  highlight  the  discussions  and  results  of  the  respective  working 
groups  and  provides  an  overall  workshop  summary. 


1.  WORKING  GROUP  1:  USERS  AND  SUPPORTERS 

1.1  PROBLEMS 

All  the  systems  highlighted  in  the  case  studies  rely  upon  “data  collection”  to  form 
a  feedback  loop.  This  feedback  loop  is  essential  for  the  integrated  diagnostic  system  to 
be  successful.  The  working  group  noted  that  DoD’s  experience  in  collecting  data  from 
flight  line  operations  has  mixed  results.  Hence,  integrated  diagnostic  systems  may  be 
vulnerable  because  of  the  need  for  an  effective  feedback  loop.  Many  of  the  presenters 
spoke  of  the  difficulty  of  updating  the  integrated  diagnostic  system  with  on-going  data 
collection. 

A  clear  definition  of  what  integrated  diagnostics  is  must  be  articulated.  Some 
users  assert  that  the  DoD  has  integrated  diagnostic  now.  Common  terms  and 
understanding  are  essential  to  minimize  risk  of  not  obtaining  what  the  DoD  asks  from  a 
contractor.  The  consensus  of  the  working  group  was  that  integrated  diagnostics  is  the 
application  of  technology  to  system  restoration  that  improves  the  process  and  is 
characterized  by  automation,  built-in-test,  expert  systems,  feedback  loops,  information 
delivery  system  and  data  collection. 

The  data  collection  effort  cited  above  interplays  with  the  fact  that  weapon 
systems  evolve.  If  an  integrated  diagnostic  approach  requires  this  feedback,  then 
configuration  control  is  critical  to  maintaining  a  valid  integrated  diagnostic  system.  An 
evolving  weapon  system  baseline  may  adversely  impact  the  data  collection  capability  as 
well  as  the  integrated  diagnostic  accuracy. 

An  ingredient  to  successful  integrated  diagnostic  system  effectiveness  is  the 
failure  modes-effects  analysis  (FMEA).  However,  several  presenters  noted  that  no 
matter  how  skillful  and  thorough  the  engineering  effort  to  predict  a  priori  failure  modes 
and  the  observable  effects,  FMEA  is  only  a  starting  point.  Routine  data  collection  and 
diagnostic  system  improvement  effort  is  required  to  keep  the  system  robust  and  accepted 
by  the  users.  This  reality  introduces  risk  into  the  integrated  diagnostic  concept. 

Adding  integrated  diagnostic  to  a  weapon  system  could  increase  complexity  and 
decrease  reliability.  As  will  be  noted  in  recommendations  below,  this  tradeoff  must  be 
reviewed  when  considering  the  overall  issue  of  integrated  diagnostic.  Note  that 
Lufthansa  refused  to  add  certain  sensors  to  their  engines  because  of  the  resulting 
decrease  in  reliability. 

The  thrust  of  the  workshop  and  tenor  of  the  presenters  were  to  design  integrated 
diagnostic  into  a  system  at  its  conception.  However,  declining  DoD  budgets  may  reduce 
the  chance  to  use  integrated  diagnostic  on  new  starts  just  because  there  will  be  so  few. 
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Consequently,  much  of  the  existing  equipment  on  the  ramp,  in  the  field,  or  at  sea  will  be 
with  us  for  a  long  time.  The  viability  of  integrated  diagnostic  may  rest  with  its  ability  to  be 
retrofit  to  all  that  existing  equipment.  Can  it  be  retrofit  or  must  it  be  part  of  the  initial 
design?  The  results  of  the  XMAN  (see  Appendix  C)  project  may  answer  this  question  and 
suggest  decisions  on  integrated  diagnostic’s  future. 

Cultural  changes  for  the  users  represent  potential  problem  areas.  As  the  XMAN 
presentation  indicated,  the  maintainers  used  XMAN  and  then  promptly  consulted  their 
old  paper  technical  data  to  confirm  the  recommended  corrective  action.  Integrated 
diagnostic  does  not  obviate  the  need  for  reliability. 

1.2  WAR  VERSUS  PEACETIME 

An  integrated  diagnostic  system  must  not  require  a  deploying  war  fighting  unit  to 
remain  linked  to  some  central  data  processing  facility.  A  unit  must  be  able  to  stand  alone 
and  fight  with  little  or  no  decrease  in  system  reliability  or  maintainability.  The  complexity 
of  war  should  not  be  further  encouraged  by  making  successful  maintenance  depend  on  a 
cosmic  data  system  that  may  be  unusable  or  unavailable  during  war.  After  the  initial  bed 
down  and  theater  status  stablizes,  data  links  can  be  re-established.  But  these  diagnostics 
related  data  links  should  not  be  a  necessity  when  using  the  systems  to  fight. 

1.3  BARRIERS 

The  lack  of  a  common  definition  for  integrated  diagnostics  has  spawned 
uncommon  approaches  and  equipments.  This  may  impede  the  introduction  of  new 
diagnostics  capabilities,  particularly  if  retrofit  is  the  most  productive  way  to  gain  the 
advantages  of  integrated  diagnostic.  Proliferation  complicates  and  discourages. 

1.4  BENEFITS 

Clearly,  the  presenters  made  the  case  for  reduced  manpower  to  maintain  systems 
with  integrated  diagnostic.  The  presenters  also  highlighted  opportunities  to  reduce  the 
system  life  cycle  costs. 

1.5  RECOMMENDATIONS 

The  workshop  participants  recommend  that  integrated  diagnostics  be  designed 
into  weapon  systems,  but  only  after  results  of  some  of  the  pilot  programs  and  current 
integrated  diagnostic  system  are  analyzed  and  true  costs  and  benefits  derived.  Having 
been  sold  “bills  of  goods”  before,  the  DoD  should  wait  for  tangible  results  to  be  visible. 
A  large  number  of  integrated  diagnostic  systems  are  under  development  and/or  near 
completion  and  use.  The  following  are  working  group  comments  relative  to  this 
recommendation: 


Look  for  other  ways  to  achieve  improved  reliability  and  note  that  systems  that 
break  infrequently  may  not  need  an  expensive,  high-risk,  data-sensitive 
integrated  diagnostic  system. 

Analyze  tradeoffs  between  integrated  diagnostic  and  basic  reliability 
improvements. 

Look  at  eliminating  a  level  of  maintenance,  e.g.,  depot  level  maintenance. 

Look  at  Deming’s  concepts  of  Total  Quality  Management.  (See  Mary  Walton’s, 
The  Deming  Management  Method,  New  York,  Perigee  Books,  1986.) 

Keep  the  status  quo  pending  evidence  that  integrated  diagnostic  is  not  a  flash  in 
the  pan. 

Proof  of  integrated  diagnostic  retrofit  opportunities  are  needed.  XMAN  may  be 
an  appropriate  program  to  watch. 


2.  WORKING  GROUP  2:  ACQUISITION  AND  SYSTEM 
DEVELOPMENT 

2.1  INTRODUCTION 

This  section  provides  the  results  of  discussion  within  the  Working  Group  on 
Acquisition  and  System  Development.  This  working  group  v/as  tasked  to  address  the 
following  ideas  and  questions: 

a.  Differences  between  commercial  and  DoD, 

b.  Lessons  learned,  benefits  and  barriers,  and 

c.  Application  opportunities. 

2.2  GENERAL 

The  working  group  observed  that  the  DoD  is  currently  experiencing  an 
interaction  between  technological  and  social  forces.  Advancements  in  technology  are 
allowing  the  rapid  transition  from  simple  monitoring  systems  into  highly  integrated 
diagnostic  systems.  However,  the  effective  implementation  of  integrated  diagnostic 
techniques  also  requires  integration  across  a  support  infrastructure  that  is  very 
functionally  oriented.  Currently  the  ability  of  this  structure  to  effectively  handle  the 
interaction  among  the  functions  is  poor.  Although  the  infrastructure  is  moving  toward 
greater  functional  integration,  it  is  occurring  at  a  much  slower  pace  than  desirable  to  keep 
up  with  technology.  Section  11.3  contains  a  discussion  of  the  differences  between 
commercial  and  military  systems,  the  lessons  to  be  learned  and  how  the  DoD  can  apply 
these  lessons. 

2.3  DIFFERENCES 

The  following  paragraphs  highlight  the  basic  differences  between  various 
requirements  and  needs  of  commercial  and  military  systems. 

a.  Threat  vs.  Market  Share.  The  major  force  behind  the  development  of  military 
systems  is  the  current  and  projected  threat  from  the  weapon  systems  of 
potential  adversaries.  These  threats  are  translatable  into  specific  capabilities 
that  must  be  countered.  In  contrast,  the  commercial  world  is  largely  driven  by 
the  need  to  capture  that  share  of  the  market  that  is  necessary  to  derive  an 
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acceptable  profit.  In  the  commercial  environment,  if  a  company  recognizes 
that  it  cannot  meet  the  challenges  of  a  particular  product  line,  the  company 
may  withdraw  from  that  competitive  environment  and  direct  energies 
elsewhere. 

b.  Weight  and  Volume.  Military  systems  frequently  have  more  severe  weight 
and  volume  constraints  than  commercial  systems.  This  characteristic  derives 
itself  from  the  threat,  the  typical  need  for  US  military  systems  to  deploy  and 
operate  over  long  distances,  and  meet  a  threat  that  is  typically  closer  to  its 
internal  lines  of  communication.  One  of  the  major  fall-outs  of  this  is  that 
commercial  systems  can  frequently  incorporate  greater  redundancy  in  order 
to  delay  required  maintenance  actions  to  a  point  of  convenience.  In  the 
military  environment,  component  redundancy  is  sometimes  incorporated  to 
reduced  unscheduled  combat  maintenance.  However  in  peacetime,  the 
maintenance  workforce  is  generally  tasked  to  keep  all  redundant  items  in 
working  condition.  Thus,  the  redundancy  actually  adds  to  the  peacetime 
workload. 

c.  Budget  vs.  Requirements.  In  a  tight  budget  environment,  the  military  Services 
may  place  emphasis  on  “rubber  on  the  ramp”  over  effective  support.  This  is 
based  on  the  argument  that  selected  spares  and  trained  people  can  be 
produced  faster  than  the  aircraft,  ships  and  ground  systems  during  a  crisis 
situation.  Although  the  military  users  may  be  dissatisfied  with  a  less  effective 
support  system,  they  have  few,  if  any,  alternatives.  In  the  commercial  world, 
the  producers  must  maintain  customer  confidence  and  satisfaction  if  they  are 
to  remain  in  business.  This  tends  to  insure  that  the  integrated  diagnostics  and 
other  support  consideration  are  not  eroded  in  the  overall  commercial  sector 
budget  decision  process. 

d.  Product  Life.  The  military  environment  is  characterized  by  much  longer 
product  life  cycles.  The  military  environment  takes  7-10  years  to  develop  a 
new  weapon  system  while,  on  average,  the  commercial  development 
environment  is  much  shorter  with  many  products  only  requiring  2-4  years.  The 
operating  life  in  the  military  ranges  from  20  to  30  years  with  upgrades.  In  the 
commercial  environment,  the  system  life  is  variable  and  is  driven  by 
technology,  customer  expectations,  and  profit  motivations. 

2.4  LESSONS  LEARNED,  BENEFITS,  AND  BARRIERS 

The  following  paragraphs  are  based  on  the  shared  experiences  of  the  working 
group  participants  and  highlighted  their  perception  of  lessons  learned,  the  benefits,  and 


the  barriers  to  integrated  diagnostic  system  implementation, 
a.  Lessons  Learned: 

(1)  Funding.  Integrated  diagnostics  and  other  support  requirements  must 
have  adequate  management  commitment  and  funding  for  the  life  of  the 
program. 

(2)  Early  Design  Definition.  The  development  of  an  effective  integrated 
diagnostics  capability  must  be  accomplished  concurrently  with  the  product 
design.  It  is  very  difficult  to  retrofit  integrated  diagnostics  onto  an  existing 
product.  Equally  important,  early  emphasis  must  be  placed  on  working 
the  support  infrastructure  to  be  compatible  with  the  integrated  diagnostic 
approaches  that  are  taken.  This  will  require  the  in-depth  involvement  of 
the  users  and  supporters  as  both  the  design  and  its  integrated  diagnostics 
features  are  developed. 

(3)  Rapid  Prototyping.  While  early  definition  of  integrated  diagnostics 
requirements  and  solutions  should  be  emphasized,  it  is  also  recognized 
that  integrated  diagnostics  is  an  evolving  technology  that  still  requires  a 
certain  amount  of  trial  and  error  in  real  world  applications.  It  is  very 
doubtful  that  a  fully  integrated  diagnostics  capability  could  be  adequately 
specified  prior  to  the  start  of  product  development.  An  evolutionary 
strategy  incorporating  rapid  prototyping  to  improve  realism  of  integrated 
diagnostic  requirements  appears  to  hold  the  most  promise.  Within  this 
strategy,  the  users  and  supporter  should  be  provided  the  opportunity  to 
gain  some  early  experience  with  the  systems  while  there  remains  the 
chance  to  influence  the  design.  The  system  should  then  be  evolved  to 
satisfy  their  needs.  Finally,  the  system  must  be  adequately  tested  prior  to 
delivery  to  operation  units. 

(4)  Local  versus  Central  Capabilities.  The  architecture  that  provides  for  the 
best  long-term  use  of  integrated  diagnostics  resources  appears  to  have  a 
local  data  collection  and  limited  information  retention.  However,  it  must 
provide  for  full  automated  analysis  capability  with  all  significant 
information  flowing  into  a  central  repository.  It  is  from  a  central  location 
that  the  continued  refinement  of  the  diagnostic  analysis  and  capability  is 
accomplished. 

(5)  “Not  Invented  Here”  Syndrome.  The  “Not  Invented  Here”  syndrome 
needs  to  be  recognized  early  and  attacked  with  great  vigor.  The  effective 
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implementation  of  integrated  diagnostics  will  cross  over  many  traditionally 
separate  functional  areas.  An  early  strategy  needs  to  be  developed  to 
anticipate  social/political/organizational  problem  areas  and  to  gain  the 
support  of  the  associated  individuals. 

b.  Benefits 

The  primary  benefits  for  DoD  systems  from  integrated  diagnostics  is 
increased  availability  while  decreasing  the  support  needs.  The  decrease  in 
support  needs  includes  reduced  burdens  in  the  logistics  areas  such  as  spares, 
support  equipment,  people,  and  manpower  specialization  training. 

c.  Barriers 

(1)  Major  Weapon  System  Dependence.  It  generally  takes  a  major  weapon 
system  development  to  drive  significant  advancements  in  the  support 
infrastructure.  Major  weapon  systems  are  usually  highly  complex  and 
have  the  broadest  impacts  on  the  support  structure. 

(2)  Poor  Contractor  Incentives.  DoD  business  strategies  currently  do  not 
provide  good  contractor  incentives  toward  integrated  diagnostics.  This 
primarily  is  due  to  the  fact  that  there  is  no  effective  means  for  a  contractor 
to  share  directly  in  the  benefits  accrued  to  the  government  from  an 
effective  integrated  diagnostics  capability.  In  fact,  exactly  the  opposite 
occurs.  If  the  integrated  diagnostics  capability  is  successful,  then  the 
contract  will  probably  sell  less  spare  items  to  the  government. 
Furthermore,  it  also  adds  complexity  to  the  system.  Thus,  yielding  greater 
cost,  schedule,  and  performance  risks  for  the  contractor. 

(3)  Functionally  Oriented  Infrastructure.  As  stated  earlier,  the  effective 
implementation  of  integrated  diagnostic  techniques  requires  integration 
across  a  support  structure  that  is  very  functionally  oriented.  Currently,  the 
ability  of  this  structure  to  effectively  handle  the  interaction  '-><*tween  the 
functional  areas  is  poor.  This  is  basically  a  social  problem  that  effects 
acquisition  communities  as  well  as  the  operational  and  support 
communities.  Although  the  infrastructure  is  moving  toward  greater 
functional  integration,  it  is  occurring  at  a  much  slower  pace  than  desirable 
to  keep  up  with  technology. 

(4)  Quantifiable  Requirements.  Currently,  there  is  a  lack  of  quantifiable, 
easily  measurable,  integrated  diagnostics  requirements.  Traditional 
measures  of  supportability  provide  for  such  factors  as  quantity  of  spares, 
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support  equipment,  people,  skills,  etc.  Similarly,  operational  measures 
include  items  such  as  failure  rates,  fix  rate,  mission  completion  success 
probabilities,  stories  generated,  etc.  However,  suitable  measures  of 
integration  effectiveness  across  the  traditional  measures  do  not  exist. 

(5)  Competition  and  break  out.  The  effective  implementation  of  integrated 
diagnostics  requires  a  true  systems  engineering  approach.  This  requires 
that  the  contractor  have  broad  system  design  and  management 
responsibility  to  ensure  the  achievement  of  a  long  term  integrated  systems 
approach.  This  strategy  is  in  direct  contrast  to  the  prevailing  government 
strategies  of  competition  and  break  out.  This  strategy  reduces  concurrent 
engineering  analysis  through  lack  of  investment  in  necessary  methods, 
tools  ,  and  information  standards. 

(6)  Specification  and  Standards.  There  are  already  too  many  specifications 
and  standards  within  DoD.  However,  engineering  information  standards 
which  would  enable  communication  across  contractors  and  Government 
do  not  exist  to  support  concurrent  diagnostic  design.  The  success  of 
integrated  diagnostics  implementation  is  best  served  by  more  logic  and 
reason  with  less  restrictive  standards. 

2.5  APPLICATION  OPPORTUNITIES 

The  characteristics  of  good  application  opportunities  are  those  that  lower  DoD 
costs  while  providing  higher  contractor  profits.  They  should  be  aimed  at  high  payoff 
areas  that  are  reasonably  under  the  control  of  one  implementing  office. 

a.  Rapid  Prototyping.  As  discussed  above,  the  evolutionary  strategy 
incorporating  rapid  prototyping  should  be  used.  This  strategy  should  provide 
demonstrated  concepts  before  writing  detailed  specification  and  standards.  It 
should  be  geared  to  proving  successes  on  a  small  scale  and  preferably 
implemented  on  existing  systems  in  order  to  show  immediate  benefits.  The 
proven  concepts  should  be  used  to  gain  broad-based  support  in  both  DoD  and 
industry  such  that  demonstrated  integrated  diagnostics  concepts  can  be 
expanded  in  scale. 

b.  Industry  Incentives.  New  approaches  to  incentivize  contractors  toward  the 
development  of  effective  integrated  diagnostic  capabilities  need  to  be 
developed. 

c.  Industry  and  Government  Forums.  The  expanded  use  of  informal  industry 
and  government  forums  is  encouraged.  These  will  help  promote  the  exchange 


of  ideas  and  the  development  of  concepts  outside  of  competitive  program 
office  environments  and  outside  of  the  traditional  staff  through  the  System 
Program  Office  to  contractor  relationship. 

2.6  DEFINITION  OF  INTEGRATED  DIAGNOSTICS 

Working  group  discussions  focused  on  the  many  meaning  and  definitions  of 
“Integrated  Diagnostics”  used  throughout  the  workshop.  The  participants  observed  that 
Integrated  Diagnostics  was  used  interchangeably  as  (1)  a  process  and/or  approach,  (2)  a 
series  contractual  needs  or  requirements,  and  (3)  as  a  measure  of  diagnostics 
capabilities.  The  working  group  recommended  that  a  consistent  set  of  definitions  and 
terms  be  developed  to  address  each  of  these  specific  areas.  The  consensus  of  the  working 
group  was  that  the  following  NSIA  definition  that  referred  to  Integrated  Diagnostics  was 
an  excellent  first  step  towards  a  standard  definition: 

Integrated  diagnostics  is  defined  as  a  structured  design  and  management  process 
to  achieve  the  maximum  effectiveness  of  a  weapon  system’s  diagnostic  capability 
by  considering  and  integrating  all  related  pertinent  diagnostic  elements.  The 
process  includes  interfaces  between  design,  engineering,  testability,  reliability, 
maintainability,  human  engineering,  and  logistic  support  analysis.  The  goal  is  a 
cost-effective  capability  to  detect  and  unambiguously  isolate  all  faults  known  or 
expected  to  occur  in  weapon  systems  and  equipment  in  order  to  satisfy  weapon 
system  mission  requirements. 


3.  WORKING  GROUP  3:  ENGINEERING  TECHNICAL  DATA  AND 
STANDARDS 

3.1  OVERVIEW  OF  GROUP  DISCUSSIONS 

Group  3  discussions  concentrated  initially  on  determining  when  to  begin  including 
integrated  diagnostics.  It  was  decided  that  the  integrated  diagnostics  requirement  was  to 
be  included  in  the  Tentative  Operational  Requirement  and  certainly  no  later  than 
Defense  Acquisition  Board  (DAB)  Milestone  I.2  This  would  assure  that  integrated 
diagnostic  would  be  included  from  concept  definition  through  development  as  part  of  the 
Functional,  Allocated,  and  Production  Baselines. 

The  working  group  discussed  the  various  diagnostic  elements  (built-in-test,  test 
equipment,  technical  information,  training,  etc.).  Discussions  focused  on  how  the 
diagnostics  elements  could  be  specified  and  included  in  development  specifications  and 
baselines.  The  working  group  felt  that  test  equipment  implementation  could  not  be 
standardized  due  to  various  manufacturer’s  unique/proprietary  configurations.  However, 
it  was  felt  that  test  information  interfaces  could  be  standardized. 

The  working  group  identified  a  number  of  engineering  information  interfaces  that 
need  to  be  considered  in  terms  of  integrated  diagnostics.  These  include  drawings, 
specification,  technical  manuals,  Logistics  Support  Analysis  (LSA),  Computer  Aided 
Engineering  (CAE),  Computer  Aided  Design  (CAD),  and  Support  Equipment 
Requirements  Document  (SERD). 

3.2  SUMMARY  OF  IDENTIFIED  CONCERNS/PROBLEMS 

The  working  groups  identified  the  following  concerns: 

a.  There  is  not  a  uniform  understanding  of  integrated  diagnostics  in  DoD, 
industry,  and  academia. 

b.  There  will  be  a  considerable  impact  on  maintenance  and  maintenance 
training  as  a  result  of  integrated  diagnostics  and  it  needs  to  be  addressed. 


2.  Department  of  Defense  Directive  5000. 1 ,  Major  and  Non-Major  Defense  Acquisition  Programs. 

September  1,  1987. 
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c.  The  effects  of  integrated  diagnostics  on  equipment  weight,  configuration, 
environment,  human  factor^  etc.,  need  to  be  addressed. 

d.  Operators  and  maintainers  must  avoid  becoming  complacent  and  expecting 
equipment  to  be  overly  self-sufficient. 

e.  The  effects  on  traditional  life  cycle  support  need  to  be  addressed. 

f.  Engineering  information  interfaces  are  not  standardized  for  integrated 
diagnostics  and  this  inhibits  design  analysis  across  weapon  systems,  and 
subsequent  analysis  during  operation. 

3.3  WORKING  GROUP  FINDINGS 

The  working  group  was  unanimously  receptive  to  integrated  diagnostics.  Most 
group  members  could  recite  examples  of  instances  where  many  hours  were  expended  and 
good  hardware  was  replaced  in  attempts  to  localize  failures. 

3.4  PRELIMINARY  RECOMMENDATIONS 

The  working  group  identified  the  following  preliminary  recommendations: 

a.  Require  integrated  diagnostics  early  in  a  program,  i.e.,  durir.g  concept 
definition. 

b.  Include  integrated  diagnostics  as  part  of  contract  requirements,  by  integral  ng 
the  needs  into  requirements  (i.e.,  avoid  separate  line  items.) 

c.  Revise  and  consolidate  current  standards  to  include  integrated  diagnostics. 

d.  Develop  engineering  information  standards  to  support  assessment  of 
integrated  diagnostics  performance. 

e .  Maintenance  and  maintenance  training  needs  must  be  addressed. 
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4.  WORKING  GROUP  4:  RESEARCH  AND  DEVELOPMENT 

4.1  SUMMARY  OF  IDENTIFIED  CONCERNS/PROBLEMS 

The  workshop  group  expressed  concern  that  there  is  a  lack  of  a  clear  definition  of 
integrated  diagnostics.  Additionally,  they  expressed  fear  that  integrated  diagnostics 
would  be  treated  as  one  more  great  idea  doomed  to  inertia. 

4.2  WORKING  GROUP  FINDINGS 

The  working  group  findings  and  opinions  are  summarized  in  the  following: 

a.  Is  it  feasible  to  acquire  and  apply  integrated  diagnostics? 

The  working  group  concluded  that  it  is  feasible  to  acquire  and  apply 
integrated  diagnostic  approach.  However,  integrated  diagnostics  must  be 
required  up  front  with  clearly  articulated  specifications.  Furthermore,  new 
design  tools  may  be  required. 

b.  Is  it  feasible  to  acquire  integrated  diagnostics  when  multiple  contracts  are 
involved? 

The  working  group  concluded  that  it  is  feasible;  however,  the  risks  are 
probably  reduced  if  only  one  contractor  is  involved.  Furthermore,  the  basis 
for  contractor  source  selection  and  award  may  become  very  important  if  DoD 
plans  to  acquire  integrated  diagnostics  system  as  a  package.  The  question  of 
how  to  handle  non-developed  items  would  also  need  to  be  addressed. 

c.  What  changes  might  be  required  to  implement  integrated  diagnostics  concepts 
on  DoD  systems? 

A  number  of  potential  changes  to  acquisition  and  support  policies  and 
procedures  may  be  required,  including  the  following  areas: 

(1)  Institutional  change 

(2)  Basis  for  award 

(3)  Metrics  and  measurement  techniques 


(4)  Testability  requirements 

(5)  DoD  policy  and  standards 

Technology  strategic  objectives  that  may  influence  integrated  diagnostics 
systems'  success  include: 

(1)  Attack  technical  barriers  in  integrated  diagnostics  (e.g.  Line  Replaceable 
Unit  (LRU)  fault  isolation  capability). 

(2)  Prognostics  (mechanical  and  electrical) 

(3)  Electronic  manuals 

(4)  Data  collection  and  feedback 

(5)  Centralized  diagnostics  system 

(6)  Interoperable  engineering  information  exchange  and  repository. 

(7)  Steering  committee 

The  following  are  potential  R&D  related  areas  that  might  have  an  influence 
on  successful  integrated  diagnostics  system  applications: 

(1)  OSD  objectives  and  policies  (life  cycle  cost  reduction,  increased 
availability  and  maximized  war  fighting  capability) 

(2)  Design  analysis  tools  (testability  analyzers,  fault  simulators,  design 
checkers,  etc.) 

(3)  Methods,  processes,  metrics 

(4)  Design  center  for  analysis 

(5)  Independent  R&D 

(6)  Design  requirements  and  specifications  (reasonable) 

(7)  Pilot  programs  and  prototyping 

(8)  Architectures 

(9)  End  user  involvement 

(10)  Technical  base  (e.g.,  neural  networks,  AI) 

(11)  Adaptable,  reconfigurable  system  architecture,  design,  and  maintenance 
strategy  and  implementation. 


4.3  PRELIMINARY  RECOMMENDATIONS 

The  working  groups  all  agreed  that  tools  were  required;  however  the  process  of 
identifying  specific  tools  requires  a  longer,  more  technically  oriented  meeting. 
Additionally,  the  group  felt  that  organization  methods  and  processes  need  to  be 
developed  in  the  area  of  integrated  diagnostics.  Furthermore,  the  working  group  felt  that 
the  workshop  was  an  excellent  forum  for  the  exchange  of  ideas.  The  group  recommends 
that  these  forums  should  be  continued  and  emphasis  should  be  placed  on  additional 
government  support  to  continue  the  development  of  an  integrated  diagnostics  support 
infrastructure. 

The  working  group  also  felt  that  a  DoD  focal  point  for  integrated  diagnostics 
would  be  beneficial  and  recommended  that  OASD(P&L)/WSIG  assume  this  role.  In  that 
capacity,  they  could  establish  a  listing  of  who’s  who  in  integrated  diagnostics  as  well  as 
serve  as  a  centralized  clearinghouse  that  could  periodically  document  and  distribute  the 
results  of  the  integrated  diagnostics  pilot  demonstration  programs.  The  working  group 
recommended  that  WSIG  consider  using  an  electronic  bulletin  board  to  facilitate  the 
information  flow. 
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5.  WORKSHOP  SUMMARY 


The  following  observations  represent  a  composite  of  the  combined  analyses  of  the 
case  studies  as  well  as  the  results  of  both  the  working  group  and  full  workshop  discussion 
forums.  These  observations  are  divided  into  seven  major  categories:  Definition  of 
Integrated  Diagnostics,  System  Design  Approach,  Commitment  to  Diagnostics 
Requirements,  Data  Analyses  and  Feedback,  Contracting  Policy  and  Approaches, 
Research  and  Development  Opportunities,  and  Diagnostic  Models  and  Design  Tools. 
Summaries  of  individual  working  group  discussions  are  located  in  Sections  1-4  of 
Appendix  D. 

5.1  DEFINITION  OF  INTEGRATED  DIAGNOSTICS 

Throughout  the  workshop,  participants  emphasized  the  need  for  a  clear  definition 
of  integrated  diagnostics.  They  found  that  the  term  “integrated  diagnostics”  was  used  in 
several  different  contexts  and  often  with  different  meanings. 

For  example,  the  term  “Integrated  Diagnostics”  was  used  (1)  to  represent  a 
structured  design  process  that  integrates  all  related  pertinent  diagnostics  elements,  (2)  to 
represent  an  acquisition  approach  that  develops  and  acquires  various  diagnostics 
elements  as  a  package,  and  (3)  to  represent  a  deliverable  system  (or  subsystem)  that 
integrates  diagnostics  elements. 

The  workshop  participants  could  not  agree  on  a  specific  definition  for  “Integrated 
Diagnostics”.  However,  there  was  general  consensus  that  integrated  diagnostics  includes 
some  application  of  technology  to  improve  the  system  restoration  process  and  is 
characterized  today  by  automation,  BIT,  expert  systems,  feedback  loops,  information 
delivery  systems  and  data  collection. 

5.2  SYSTEM  DESIGN  APPROACH 

The  individual  steps  of  the  system  design  process  leading  to  the  development  and 
incorporation  of  enhanced  diagnostics  capabilities  were  found  to  be  virtually  identical 
whether  the  process  was  targeted  towards  a  new  design  or  was  focusing  on  an  existing 
system  application.  The  opportunity  to  deliver  an  effective  diagnostic  capability  for  the 
system  undergoing  the  design  process  was  a  function,  of  four  factors  that  relate  to  the 
maturity  of  the  target  system:  technology,  configuration  constraints,  system  trade-offs, 


and  maintenance  data  availability. 

The  working  groups  observed  that  the  viability  of  integrated  diagnostics  may  rest 
with  its  ability  to  be  retrofited  to  ail  that  existing  equipment.  The  case  studies  supported 
the  conclusion  that  it  is  indeed  viable  to  retrofit  diagnostic  capabilities  into  existing 
systems. 

5.3  COMMITMENT  TO  DIAGNOSTIC  REQUIREMENTS 

Participants  agreed  that  a  strong  DoD  commitment  to  developing  and  enhancing 
the  diagnostics  capabilities  of  weapons  systems  is  essential  to  improving  effectiveness  in 
terms  of  increased  system  availability  and  reduced  maintenance  burdens.  Furthermore, 
it  was  viewed  as  essential  that  this  commitment  be  sustained  throughout  the  product  life 
cycle. 

The  rationale  and  justification  for  improving  the  diagnostics  capabilities  of  the 
new  and  evolving  weapon  systems  must  originate  from,  and  be  sustained  based  on,  the 
operational  needs  and  requirements  of  the  using  and  maintaining  communities.  In  order 
to  develop  a  sustained  commitment: 

•  DoD  needs  to  better  understand  the  basic  needs  and  requirements  for  integrated 
diagnostics,  especially  those  that  span  the  product  life  cycle. 

•  DoD  needs  to  invest  in  models  and  tools  that  help  forecast  the  benefits  of 
integrating  diagnostics  elements  and  assist  in  determining  integrated  diagnostics 
requirements. 

•  The  logistics  organizations  and  systems  must  facilitate  developer  involvement  in 
support  of  the  maturing  diagnostics  system. 

•  Cultural  change  resulting  from  the  introduction  of  integrated  diagnostics  systems 
may  be  significant.  User  confidence  must  be  acquired  incrementally  through 
verification,  and  integrated  diagnostics  implementation  plans  must  recognize  this 
condition. 

5.4  DATA  ANALYSES  AND  FEEDBACK 

Systems  with  highly  effective  diagnostic  capabilities  typically  employed  effective 
maintenance  data  collection,  centralized  data  analyses,  and  rapid  feedback  of  critical 
information  to  both  the  design  and  maintaining  communities.  Attributes  that 
characterized  effective  maintenance  data  collection  included  on-line  data  collection, 
limited  opportunity  for  human  error,  and  electronic  presentation  and  interaction  with  the 
system  maintainer. 


Centralized  data  analysis  was  observed  to  be  critical  for  improving  diagnostic 
maturation.  It  increases  the  population  sample  for  analyses,  eliminates  latency  of 
maintenance  data,  and  provides  opportunities  to  develop  statistically  accurate  prognostic 
algorithms.  The  workshop  participants  also  noted  that  as  new,  more  reliable  systems  are 
introduced,  it  will  physically  take  longer  to  collect  the  necessary  data  for  formulating 
enhancements  to  the  design  and/or  diagnostic  approaches  unless  centralized  data 
analysis  concepts  are  adopted. 

Effective  data  feedback  loops  to  the  maintainers  and  designers  are  essential  to 
successful  integrated  diagnostics  implementation.  However,  the  effectiveness  and 
accuracy  of  field  maintenance  data  collection  for  DoD  systems  are  generally  less  capable 
than  those  illustrated  in  the  case  studies.  DoD  field  units  currently  collect  enormous 
amounts  of  data  that  are  not  being  analyzed  effectively.  Opportunities  to  enhance  the 
DoD  data  collection  analyses  and  feedback  abound.  The  workshop  concluded  that 
improvements  in  system  diagnostics  capabilities  may  be  hampered  if  the  DoD  relies  on 
the  current  field  data  collection  capabilities. 

Configuration  control  is  essential  and  diagnostics  is  only  one  aspect  of  systems 
operation  and  maintenance  that  requires  this  essential  capability.  Integrated  diagnostic 
systems  with  automated  data  collection  and  analysis  capabilities  can  actually  help 
automate  and  improve  the  configuration  management  and  control. 

5.5  CONTRACTING  POLICY  AND  APPROACHES 

New  approaches  for  identifying  integrated  diagnostics  needs  must  be 
incorporated  into  the  system  contractual  requirements.  Integrated  diagnostics  cannot  be 
effectively  implemented  as  multiple  line  items  for  discrete  diagnostics  elements  unless 
there  are  appropriate  interface  specifications  that  include  diagnostics  interface  needs. 
Importance  of  this  aspect  increases  if  there  are  multiple  contracts. 

The  participants  also  felt  that  it  is  feasible  to  develop  standards  for  testing  and 
data/information  interfaces,  but  the  standards  should  be  t:  ,d  to  the  minimum  level 
necessary  to  achieve  successful  implementation  across  a  common  set  of  weapon  system 
platforms  served  by  one  centralized  analysis  center.  Extensions  of  the  CALS  information 
standards  to  support  integrated  diagnostics  will  be  necessary  to  support  long  term 
diagnostic  improvement  goals. 

Support  system  environments  and  contracting  methods  must  ensure  the  proper 
balance  of  rewards,  benefits,  and  incentives  that  may  be  shared  with  contractors  over  the 
product  life  cycle.  Without  the  appropriate  combination  of  contractual  requirements, 
incentives,  and  rewards,  it  is  unlikely  that  contractor  communities  will  develop  a 


sustained  commitment  towards  developing  and  delivering  an  effective  diagnostics 
capability. 

5.6  RESEARCH  AND  DEVELOPMENT  OPPORTUNITIES 

DoD  weapon  systems  have  unique  integrated  diagnostic  needs,  driven  by 
operational  requirements  that  are  often  not  evident  in  commercial  applications.  DoD 
integrated  diagnostics  implementation  opportunities  could  be  expanded  through  research 
and  development  initiatives  that  focus  on  common  weapon  subsystem  architectures  and 
pilot  demonstration  programs  conducted  in  the  unique  DoD  operational  environments. 

In  addition,  DoD  integrated  diagnostics  methodologies  and  approaches  would 
benefit  significantly  from  research  and  development  initiatives  targeted  towards  enabling 
integrated  diagnostics  technologies.  The  following  is  a  list  of  potential  R&D  enabling 
technology  opportunities  developed  by  the  participants. 

•  Multi-function  condition  sensors. 

•  Development  of  centralized  data  collection,  analysis,  and  distribution  facilities, 
capabilities  and  methods. 

•  Maintenance  prognostics  technologies. 

•  Integrated  diagnostics  design,  synthesis,  and  assessment  tools. 

•  Diagnostics  models  (artificial  intelligence  and  rule  based). 

•  Electronic  presentation  and  authoring  tools  correlated  with  diagnostics  models. 

•  Sets  of  standardized  on/off  equipment  diagnostics  interfaces. 

•  Extensions  of  CALS  engineering  information  standards  in  product  and  test 
specification  and  description  and  other  diagnostic  interfaces. 

•  Integrated  diagnostics  architecture,  design,  and  implementation. 

5.7  DIAGNOSTIC  MODELS  AND  DESIGN  TOOLS 

DoD  generally  bases  an  initial  system’s  diagnostic  and  maintenance  procedures 
on  the  Failure,  Mode,  and  Effects,  Analyses  (FMEA)  data.  Unfortunately  this  data  is  not 
representative  of  the  final  operational  system.  No  matter  how  skillful  and  thorough  the 
engineering  effort  to  predict  a  priori  failure  modes,  the  FMEA  is  typically  inaccurate  and 
incomplete.  The  FMEA  is  a  starting  point;  however,  new  design  tools  and  diagnostic 
models  to  support  decision  trade-off  analyses  are  needed. 
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The  tools  must  support  a  methodology  for  improving  diagnostics  as  the 
information  available  changes  from  design-based  knowledge  of  the  weapon 
system  to  maintenance  history  and  experience-based  knowledge. 

The  models  and  tools  must  take  into  account  that  diagnostic  effectiveness  will 
improve  over  time,  and  the  models  must  facilitate  the  development  and  evolution 
of  appropriate  strategies  to  accelerate  this  maturation  process. 
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